Data communication method and related device
By carrying connection identifiers and data identifiers in RDMA data packets and utilizing idle QPs for data transmission, the problem of excessive memory resource consumption under large-scale connections is solved, achieving efficient data transmission and resource sharing.
Patent Information
- Application Number
- PCT/CN2024/136355
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-04-22
- Filing Date
- 2024-12-03
- Publication Date
- 2025-10-30
AI Technical Summary
In large-scale connection scenarios, existing RDMA technology requires the establishment of a specific queue pair (QP) for each connection, resulting in the consumption of a large amount of memory resources and failing to meet the needs of large-scale connections.
By including the connection identifiers of the source and destination ends, as well as the data identifier, in the header information of the data packet, data transmission is carried out using idle QPs, eliminating the dependence on specific QPs, realizing flexible data transmission, and sharing QPs to save memory resources.
It enables flexible data transfer, reduces the memory resources required to establish a specific QP, and improves data transfer efficiency and resource utilization.
Smart Images

Figure CN2024136355_30102025_PF_FP_ABST
Abstract
Description
A data communication method and related equipment
[0001] This application claims priority to Chinese Patent Application No. 202410490574.5, filed on April 22, 2024, entitled "A Data Communication Method and Related Equipment", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application relates to the field of computer communications, and more particularly to a data communication method and related equipment. Background Technology
[0003] Remote Direct Memory Access (RDMA) is a high-efficiency network communication technology that allows a computer system to directly access the memory of a remote computer over a network without the involvement of the Central Processing Unit (CPU). The advantages of RDMA include reduced CPU load, increased data transfer speed and bandwidth, and reduced network latency.
[0004] Currently, RDMA technology commonly employs a reliable connection (RC) approach. Specifically, for each connection: First, the sender and receiver each need to define a specific queue pair (QP). Second, the sender and receiver establish a connection between the QPs (e.g., via sockets, Communication Management Protocols (CMs), etc.). Finally, the message is transmitted from the send queue of the sender's specific QP to the receive queue of the receiver's specific QP, thus achieving message transmission.
[0005] However, the above method requires establishing specific QPs before transmitting messages, and messages can only be transmitted through specific QPs. In large-scale connection scenarios, the number of specific QPs can be enormous, consuming huge amounts of memory resources and failing to meet the demands of large-scale connections. Summary of the Invention
[0006] This application provides a data communication method and related equipment, which can not only flexibly realize the transmission of data packets, but also save the memory resources required to establish a specific QP.
[0007] This application provides a data communication method applied to a first network interface card (NIC) based on the Remote Direct Memory Access (RDMA) protocol. The method is executed by the first NIC, or by some components of the first NIC (e.g., a processor, chip, or chip system), or by a logic module or software capable of implementing all or part of the functions of the first NIC. The first NIC may also be called an RDMA NIC, and it is located in a first computer device containing a first processor. The first NIC can be connected to the first processor in a pluggable manner, through chip integration, or via a bus, etc. In this first aspect and its possible implementations, the method is described using the execution of the first NIC as an example.
[0008] In this method, the first network interface card (NIC) receives first information sent by the first processor. The first information includes the storage address and length of the first data. The first information can also be understood as metadata sent by the first processor to the first NIC. This process can also be understood as the transmission of first information between the first NIC and the first processor of the first computer device. Furthermore, the first data can refer to any data that an application process in the first computer device wants to send. For example, the first data can be a message generated by a communication application that needs to be sent.
[0009] The first network interface card (NIC) can also send a first message to the second NIC in the second computer device based on the first queued QP. The first message includes a first payload and a first header. The first header includes: a connection identifier of the first source end, a connection identifier of the first destination end, and an identifier of the first data. The first payload is used to carry the first data. The identifier of the first data can uniquely identify the first data, and the state of the first QP is idle.
[0010] The first payload can carry the entire first data or a portion of the first data. Furthermore, the connection identifier at the first source end and the connection identifier at the first destination end jointly identify a transmission channel.
[0011] In this scheme, the first network interface card (NIC) can send data packets through an idle first QP (Quick Pointer). The data packet header includes the source connection identifier, the destination connection identifier, and the message sequence number of the payload data. The source and destination connection identifiers jointly identify a transmission channel, meaning the connection carrier is the same entity within the data packet, eliminating the need for a specific QP. In other words, the data packet can carry transmission channel information without pre-establishing a connection to a specific QP before sending the message; that is, the transmission channel information is unbound from a specific QP. This approach not only allows for flexible data packet transmission but also saves memory resources required to establish a specific QP. Furthermore, because the transmission channel information is unbound from a specific QP, this method can be called a message-based short connection compared to existing long connections with fixed QPs. For example, after application process 1 transmits a message using QP1, QP1 is set to idle, allowing application process 2 to use QP1 to transmit messages, thus enabling multiple application processes to share a single QP.
[0012] In one possible implementation, before sending the first packet to the second network interface card (NIC) in the second computer device based on the first QP, the first NIC can first determine the first idle QP in the first queue pair group (QPG). After determining the first QP, the state of the first QP is changed from idle to non-idle.
[0013] The process of determining the first QP by the first network card can be either to directly determine the first QP from the first QPG, or to create a new idle QP for the first QPG and use the idle QP as the first QP.
[0014] In addition, the first data in this method can be identified by the message sequence number (MSN) alone, or by the MSN and the queue pair group number (QPGN) of the first QPG.
[0015] In this scheme, the first network interface card (NIC) can identify an idle first QP within the first QPG, and then adjust the state of the first QP to non-idle. This adjustment prevents other application processes from using the non-idle first QP. Of course, if the state of the first QP is subsequently adjusted to idle, other application processes can use that idle QP to transmit data.
[0016] In one possible implementation, the first information further includes first indication information, which is used to indicate that the first data is sent through the first QP.
[0017] In this scheme, the first processor, where the first network interface card (NIC) is located, determines the first available QP (Queries Pointer) and indicates the QP used by the first NIC to transmit the first data through the first indication information in the first information. Thus, the first NIC can clearly specify the QP used to transmit the first data through the first indication information.
[0018] In one possible implementation, the first network interface card (NIC) can further establish a first context for the first data, which is used to maintain the transmission status of the first data. For example, the first context for the first data includes at least one of the following: the amount of data already transmitted, and how much data remains to be transmitted. It is understood that the first context can also include an offset, which, when the first data is transmitted through multiple packets, indicates the position of the transmitted first packet within the multiple packets. Of course, after the first NIC establishes the first context for the first data, it can also delete the first context if a first preset condition is met; the first preset condition includes at least one of the following: all packets for the first data have been transmitted; or, an acknowledgment message has been received from the second NIC for all packets corresponding to the first data. It is understood that, when the first data is transmitted through multiple packets, the first preset condition could be receiving an acknowledgment message for the last packet among the multiple packets.
[0019] In this scheme, the first network interface card (NIC) maintains the transmission status of the first data through an established first context, ensuring that the first data is transmitted correctly and that the second NIC can receive the first data completely. For example, in the event of a lost packet, the first context can be used to determine which packets need to be retransmitted, guaranteeing that the second NIC can reliably receive the first data. Furthermore, promptly deleting the data context after data transmission is complete not only saves storage space but also reduces the power consumption of the first NIC.
[0020] In one possible implementation, the first network interface card (NIC) can also receive an acknowledgment message corresponding to the first message from the second NIC. This acknowledgment message indicates that the second NIC has correctly received the first message. Understandably, to ensure the first NIC can identify which message corresponds to the acknowledgment message, the acknowledgment message may also carry the three identifiers from the aforementioned first header information corresponding to the first message.
[0021] In this scheme, the first network interface card (NIC) can confirm that the second NIC has correctly received the first message through the acknowledgment message of the second message. This facilitates other operations or subsequent operations for the first NIC and reduces timeout retransmissions. For example, after receiving an acknowledgment message for a certain message, the first NIC can send subsequent messages to the second NIC to improve the transmission efficiency of the first data. As another example, after receiving an acknowledgment message for the last message, the first NIC can delete the first context, saving storage space and reducing the power consumption of the first NIC.
[0022] In one possible implementation, the first QP includes a send queue (SQ) and a completion queue (CQ); the SQ carries first information; the first network interface card (NIC) sends a first message to a second NIC in a second computer device based on the first queue QP, including: the first NIC acquiring the first data based on the SQ; the first NIC encapsulating the first data to obtain the first message; and the first NIC sending the first message to the second NIC. Furthermore, the first NIC can also send second information to a first processor based on the CQ, the second information indicating the completion of the first data transmission. Similarly, this process can also be understood as the transmission of second information between the first NIC and the first processor of the first computer device. For example, the second information could refer to an element in the completion queue (CQE).
[0023] In this scheme, after the first network card completes the transmission of the first data, it can send the second information to the first processor where the first network card is located, so that the first processor knows that the first data has been transmitted successfully, so that the application process of the first data knows that the first data has been sent correctly and can perform subsequent operations.
[0024] In one possible implementation, the second information mentioned above includes the connection identifier of the first source end.
[0025] In this scheme, by carrying the connection identifier of the first source end in the second information, the first processor can determine which application process completed the data transmission based on the connection identifier of the first source end.
[0026] In one possible implementation, the first network interface card (NIC) can also change the state of the first QP from non-idle to idle. For example, after sending the second information to the first processor, the first NIC changes the state of the first QP from non-idle to idle. Another example is that after receiving a complete acknowledgment message from the second NIC, the first NIC changes the state of the first QP from non-idle to idle.
[0027] In this scheme, after message transmission is complete, the first network interface card (NIC) can change the state of the first QP from non-idle to idle, thus facilitating the subsequent sending of other messages through the idle first QP. It can be seen that in this method, the first QP is promptly released for other message transmission after sending the message, thereby achieving message-based short connections.
[0028] In one possible implementation, after the first QP changes from non-idle to idle, the first network interface card (NIC) can also receive third information sent by the first processor. This third information includes: the storage address of the second data, the length of the second data, and the fact that the first and second data are generated by different application processes. The SQ in the first QP is used to carry this third information.
[0029] Similar to the first information, the third information can be understood as additional metadata sent by the first processor to the first network interface card (NIC). This process can also be understood as the transmission of third information between the first NIC and the first processor of the first computer device. Furthermore, the second data can refer to any data that another application process in the first computer device wants to send. For example, the second data could be a message generated by a communication application and needing to be sent.
[0030] The first network interface card can also send a second message based on the first QP. The second message includes a second payload and a second header. The second payload is used to carry the second data. The second header includes: the connection identifier of the second source end, the connection identifier of the second destination end, and the identifier of the second data.
[0031] Similarly, the second payload can carry the entire second data or a portion of the second data. Furthermore, the connection identifier at the second source end and the connection identifier at the second destination end jointly identify another transmission channel.
[0032] In this scheme, after the first network interface card (NIC) transmits the first data of an application process through the first QP, it can also transmit the second data of the same application process or the second data of other application processes through the first QP. That is, the first QP does not necessarily belong to a specific application process; different application processes can share a single QP to transmit data. This means unbinding the transmission channel information from a specific QP, thereby enabling multiple application processes to share a single QP. This method not only allows for flexible data transmission but also saves the memory resources required to establish a specific QP. It is understood that if the second data and the first data are data to be sent to the same application process, the source connection identifiers corresponding to the first data and the second data can be the same or different.
[0033] In one possible implementation, the aforementioned first data corresponds to multiple messages, and the first header information also includes an offset, which is used to indicate the position of the transmitted first message among the multiple messages. Alternatively, it can be understood that there are multiple first payloads, that is, the first data is carried by multiple first payloads, and the offset can also be understood as indicating the position of the first payload within the first data.
[0034] In this scheme, the second network card on the receiving side can correctly splice multiple payloads using the offset in the first header information.
[0035] In one possible implementation, the aforementioned first information may also include an identifier for the first data. Similarly, the first data may be identified solely by MSN, or it may be jointly identified by MSN and QPGN. Furthermore, the first information may also include a connection identifier for the first source end and a connection identifier for the first destination end.
[0036] In this scheme, the first processor of the first computer device, where the first network interface card (NIC) is located, can assign an identifier to the first data to distinguish different data. The first NIC clearly identifies the first data through first information, thereby facilitating the subsequent transmission of the first data and the establishment of the first context.
[0037] In one possible implementation, the first QP mentioned above belongs to a queue pair group first QPG, which is created by a first processor and is used to transfer data of multiple application processes on a first computer device based on RDMA.
[0038] In this scheme, multiple application processes on the first computer device can transfer data through the first QPG. For example, the first QP can transfer data for application process 1 as well as application process 2. This allows multiple application processes to share the same QPG. This method not only enables flexible data transfer but also saves the memory resources required to create specific QPs.
[0039] In one possible implementation, the aforementioned first QPG is a QPG shared by multiple application processes, including a first application process and a second application process. The first data is generated by the first application process, and the second data is generated by the second application process.
[0040] In this scheme, after the first network interface card (NIC) transmits the first data of a certain application process through the first QP in the first QPG, it can also transmit the second data of other application processes through the first QP. That is, the first QPG does not belong to any specific application process; different application processes can share the first QPG to transmit data, thus enabling multiple application processes to share the first QPG. This method not only allows for flexible data transmission but also saves the memory resources required to establish a specific QP for each application process.
[0041] In one possible implementation, the first network interface card (NIC) can also determine the identifier of the first data, which is used by the first NIC to send the first message.
[0042] In this scheme, the first network interface card (NIC) can assign an identifier to the first data itself, instead of requiring the first processor to do so, thereby reducing the power consumption of the first processor. This also facilitates the first NIC in maintaining the identifiers for each data item, making data scheduling easier.
[0043] In one possible implementation, the first information and the first header information also include the available memory address of the memory in the second computer device where the second network interface card (NIC) is located. This available memory address is used to store the first payload. For example, the first processor and the second processor can interact with the available memory in the second memory. The first processor can indicate the available memory address in the second memory to the first NIC via the first information. Thus, the first NIC can carry the available memory address in the second memory in the first header information, so that after receiving the first message, the second NIC can store the first data carried in the first payload according to the address carried in the first header information.
[0044] In this scheme, this method can also be understood as a one-sided scenario. After receiving the first message, the second network card can determine which memory block in the second memory to store the first data carried by the first payload based on the available memory address carried by the first message.
[0045] The second aspect of this application provides a data communication method. This method is executed by a first computer device, or by a component of the first computer device (e.g., a first processor, chip, or chip system), or by a logic module or software capable of implementing all or part of the functions of the first computer device. The first computer device communicates based on the Remote Direct Memory Access (RDMA) protocol. The first computer device includes a first processor and a first network interface card (NIC), or it can be understood that different computer devices communicate via RDMA protocol through NICs connected to their respective processors. For example, the first computer device can communicate via RDMA with a second NIC of a second computer device through the first NIC. The first NIC can be connected to the processor in a pluggable manner, integrated into a chip, or connected via a bus, etc. The first NIC can also be called an RDMA NIC. In this second aspect and its possible implementations, the method is described using the example of execution by the first processor.
[0046] In this method, a first processor creates a first queue pair group (QPG), which includes multiple queue pairs (QPs). The first QPG is used for transmitting data from multiple application processes via RDMA. This can also be understood as the first QPG being shared by multiple application processes on the first computer device. Furthermore, the creation of the first QPG by the first processor can be completed in one step or in multiple steps. For example, the first processor first creates the first QPG, which has a small number of QPs, and then creates new QPs for the first QPG based on data transmission needs or application process requests.
[0047] The first processor acquires first data from the first application process among multiple application processes and sends first information to the first network interface card. The first information includes the storage address and length of the first data. This first data can refer to any data (or any message) that the first application process in the first computer device wants to send.
[0048] Similarly, the first information can also be understood as the metadata sent by the first processor to the first network card. This process can also be understood as the transmission of the first information between the first network card and the first processor of the first computer device on which the first network card is located.
[0049] In this scheme, multiple application processes on the first computer device can transfer data through the first QPG. For example, the first QPG can transfer data for application process 1 as well as application process 2. This allows multiple application processes to share the same QPG. This approach not only enables flexible data transfer but also saves the memory resources required to create a specific QP for each application process.
[0050] In one possible implementation, the first processor may further determine an idle first QP among multiple QPs. The first information also includes first indication information, which instructs the first network interface card (NIC) to send first data through the first QP. After determining the first QP, the state of the first QP is changed from idle to non-idle. The first QP includes a transmit queue (SQ) and a complete queue (CQ); the SQ is used to carry the first information.
[0051] The process of the first processor determining the first QP can be either to directly determine the first QP from the first QPG, or to create a new idle QP for the first QPG and use the idle QP as the first QP.
[0052] In this scheme, the first processor can determine an idle first QP in the first QPG. On one hand, it can indicate the idle first QP to the first network card through a first indication message, so that the first network card can transmit the first data through the idle first QP. On the other hand, after determining the first QP, the first processor promptly adjusts the state of the first QP to non-idle. By adjusting the state of the first QP, other application processes cannot use the non-idle first QP. Of course, if the state of the first QP is subsequently adjusted to idle, other application processes can use the idle QP to transmit data.
[0053] In one possible implementation, the first processor can also receive second information sent by the first network card based on the CQ in the first QP. This second information indicates that the first data transmission is complete. Similarly, this process can be understood as the transmission of second information between the first network card and the first processor of the first computer device on which the first network card is located. For example, this second information could refer to CQE.
[0054] In this scheme, the first processor can confirm that the first data has been successfully transmitted through the second information, so that the first application process of the first data knows that the first data has been sent correctly and can perform subsequent operations.
[0055] In one possible implementation, the second information mentioned above includes a connection identifier of the first source end, which is associated with the first application process.
[0056] In this scheme, the first processor can determine that the data transmission of the first application process is complete by the connection identifier of the first source end.
[0057] In one possible implementation, after receiving the second information sent by the first network card, the first processor can also change the state of the first QP from non-idle to idle.
[0058] In this scheme, after message transmission is complete, the first processor can change the state of the first QP from non-idle to idle, thus facilitating the subsequent transmission of other messages through the idle first QP. It can be seen that in this method, the first QP is promptly released for other message transmission after message transmission, thereby achieving message-based short connections.
[0059] In one possible implementation, after the state of the first QP changes from non-idle to idle, the first processor can also acquire the second data of the second application process among multiple application processes; the first processor can also send third information to the first network card, the third information including: the storage address of the second data and the length of the second data, the third information being used by the first network card to send the second data through the first QP.
[0060] The third information, similar to the first information, can be understood as additional metadata sent by the first processor to the first network interface card (NIC). This process can also be understood as the transmission of third information between the first NIC and the first processor of the first computer device on which the NIC resides. Furthermore, the second data can refer to any data (or any message) that another application process in the first computer device wants to send. The SQ in the first QP is also used to carry the third information.
[0061] Similarly, the second payload can carry the entire second data or a portion of the second data. Furthermore, the connection identifier at the second source end and the connection identifier at the second destination end jointly identify another transmission channel.
[0062] In this scheme, after the first application process has sent its first data through the first QP, it can also send the second data of the same application process or the second data of a second application process through the first QP. That is, the first QP does not necessarily belong to a specific application process; different application processes on the first computer device can share a single QP to send data. This debinds the transmission channel information from a specific QP, allowing multiple application processes to share a single QP. This approach not only enables flexible data transmission but also saves the memory resources required to establish a specific QP.
[0063] In one possible implementation, the first information mentioned above further includes: a connection identifier of the first destination, which is used by the first network card to send the first data; the first processor may also receive the connection identifier of the first destination sent by the second processor.
[0064] In this scheme, the first processor and the second processor can interact with the connection identifier of the first destination. Thus, the first processor can indicate the connection identifier of the first destination through the first information, which facilitates the first network card to transmit the first data. At the same time, it also makes it convenient for the second network card to determine which application process on the second computer device needs to receive the data after receiving the first message of the first data based on the connection identifier of the first destination.
[0065] In one possible implementation, the first information further includes: a connection identifier of a first source end, which is used by the first network interface card to send the first data; the first processor may also allocate a connection identifier of the first source end to the transmission channel, which, together with the connection identifier of the first destination end, identifies the transmission channel, and the transmission channel is used to transmit the first data.
[0066] In this scheme, the first processor can allocate a first source connection identifier for the transmission channel. This not only facilitates the joint identification of a transmission channel with the first destination connection identifier, but also allows the source and destination connection identifiers in the data packet to be used together as the connection carrier, eliminating the need for a specific QP as the connection carrier. In other words, the data packet can carry transmission channel information without needing to establish a specific QP connection before sending the message; that is, the transmission channel information is unbound from a specific QP. This approach not only enables flexible data packet transmission but also saves the memory resources required to establish a specific QP.
[0067] In one possible implementation, the first processor may further determine the available memory address of the memory in the second computer device; the first information also includes the available memory address, the storage space indicated by the available memory address being used to store the first data. For example, the first processor and the second processor may interact regarding the available memory in the memory of the second computer device. The first processor can indicate the available memory address in the second memory to the first network interface card (NIC) through the first information, so that the first NIC can carry the available memory address of the memory in the second computer device in the first header information, so that after receiving the first message, the second NIC can store the first data carried by the first payload according to the address carried in the first header information. It is understood that the available memory address of the memory in the second computer device can specifically be used for reading or writing. For example, "writing" corresponds to storing the first data carried by the first payload into the storage space corresponding to the available memory address of the memory in the second computer device. Another example is "reading," which corresponds to copying the data at the available memory address of the memory in the second computer device to the temporary storage of the second NIC, thereby transmitting it to the first NIC.
[0068] In this scheme, this method can also be understood as a one-sided mode. After receiving the first message, the second network card can determine which memory block in the second computer device's memory to store the first data carried by the first payload based on the available memory address carried by the first message.
[0069] In one possible implementation, the aforementioned first information may also include an identifier for the first data. Similarly, the first data may be identified solely by MSN, or it may be jointly identified by MSN and QPGN. Furthermore, the first information may also include a connection identifier for the first source end and a connection identifier for the first destination end.
[0070] In this scheme, the first processor of the first computer device, where the first network interface card (NIC) is located, can assign an identifier to the first data to distinguish different data. The first NIC clearly identifies the first data through first information, thereby facilitating the subsequent transmission of the first data and the establishment of the first context.
[0071] The third aspect of this application provides a data communication method applied to a second network interface card (NIC) based on the Remote Direct Memory Access (RDMA) protocol. This method is executed by the second NIC, or by some components of the second NIC (e.g., a processor, chip, or chip system), or by a logic module or software capable of implementing all or part of the functions of the second NIC. The second NIC may also be called an RDMA NIC, and it is located in a second computer device containing a second memory. This second computer device is a different computer device from the first computer device containing the first NIC described in the first aspect. The second NIC can be connected to a second processor in the second computer device via plug-in connection, chip integration, or a bus, etc. In this third aspect and its possible implementations, the method is described using the execution of the second NIC as an example.
[0072] In this method, the second network interface card (NIC) receives a first message from the first NIC. The first message includes a first payload and a first header. The first header includes a connection identifier of a first source end, a connection identifier of a first destination end, and an identifier of the first data. The first payload carries the first data. The second NIC can also store the first data in a second memory.
[0073] In this configuration, the reception of the first message does not use a queue pair (QP), and the identifier of the first data can uniquely identify the first data. The first payload can carry the entire first data or a portion of the first data. The connection identifier of the first source end and the connection identifier of the first destination end jointly identify a transmission channel.
[0074] Furthermore, the second network interface card (NIC) can directly access the second memory without going through the second processor, thereby storing the received data in the second memory. There are several ways the second NIC can store the first data in the second memory. For example, the second NIC can first store the first data in its temporary storage, and then write the data from the temporary storage to the second memory after the amount of data in the temporary storage reaches a threshold or after a certain period of time since receiving the first packet. Another example is that when receiving multiple packets, the second NIC can directly store the data from each received packet into the second memory. Yet another example is that when receiving multiple packets, the second NIC can store the data from all the received packets into the second memory after receiving all the packets.
[0075] In this scheme, the header information of the data packets received by the second network interface card (NIC) carries the connection identifier of the source end, the connection identifier of the destination end, and the message sequence number of the payload data. Furthermore, the connection identifiers of the source and destination ends jointly identify a transmission channel. This means that the connection identifiers of the source and destination ends in the data packet can be used together as the connection carrier, without requiring a specific QP as the connection carrier. In other words, the data packet can carry the transmission channel information without establishing a specific QP connection before sending the message. For example, the second NIC can receive data packets without using a QP, debinding the transmission channel information from the QP. This approach not only allows for flexible data packet transmission but also saves the memory resources required to establish a QP. Moreover, since the data of the transmission channel is debinded from the QP, compared to existing long connections with fixed QPs, this method can also be called a data-based short connection.
[0076] In one possible implementation, the second network interface card (NIC) can also establish a second context for the first data. This second context is used to maintain the reception status of the first data. For example, the second context for the first data includes at least one of the following: the amount of first data already received, and the amount of data yet to be received. Understandably, the second context can also include an offset, which, when the first data is transmitted through multiple packets, indicates the position of the transmitted first packet within the multiple packets. Of course, after establishing the second context for the first data, the second NIC can also delete the second context if a second preset condition is met. The second preset condition includes at least one of the following: acknowledgment messages for all packets corresponding to the first data have been sent; or, fourth information has been sent to the second processor; the fourth information indicates that the reception of the first data is complete.
[0077] In this scheme, the second network interface card (NIC) maintains the reception status of the first data through an established second context, ensuring that the first data is correctly transmitted and completely received by the second NIC. For example, in the event of a lost packet, the second context can be used to determine which packets need to be retransmitted, guaranteeing reliable reception of the first data by the second NIC. Furthermore, promptly deleting the data context after data transmission not only saves storage space but also reduces the power consumption of the second NIC.
[0078] In one possible implementation, the second network interface card (NIC) can also send an acknowledgment message for the first packet to the first NIC. This acknowledgment message indicates that the first packet has been correctly received. Understandably, to allow the first NIC to identify which packet corresponds to the acknowledgment message, the acknowledgment message can also carry the three identifiers from the aforementioned first header information corresponding to the first packet.
[0079] In this scheme, the second network interface card (NIC) can send an acknowledgment message to the first NIC, confirming that the second NIC has correctly received the first packet. This facilitates other operations or subsequent operations for the first NIC and reduces timeout retransmissions. For example, after receiving an acknowledgment message for a packet, the first NIC can send subsequent packets to the second NIC to improve the transmission efficiency of the first data. As another example, after sending an acknowledgment message for the last packet, the second NIC can delete the second context to reduce the power consumption incurred by the second NIC in maintaining the second context.
[0080] In one possible implementation, the second network interface card (NIC) can also send a fourth message to the second processor, indicating that the first data reception is complete. Similarly, this process can be understood as the transmission of a fourth message between the second NIC and the second processor. For example, the second NIC sends the fourth message to the second processor based on CQ (Continuous Queuing). That is, this fourth message can refer to CQE (Continuous Queuing). CQ is used to transmit data reception completion information for multiple application processes on the second computer device.
[0081] In this scheme, after the second network interface card (NIC) receives the first data, it can send a fourth message to the second processor to confirm that the first data transmission is complete. This allows the application processes related to the first data to know that the first data has been correctly received and to proceed with subsequent operations. Furthermore, the second NIC sends the fourth message to the second processor through a CQ shared by multiple application processes. Compared to the method where the second NIC sends a CQE to the second processor through a specific QP, this reduces the maintenance of specific QPs.
[0082] In one possible implementation, the fourth piece of information mentioned above includes a connection identifier for the first destination. The connection identifier for the first destination is used to clarify that the first data is associated with a third application process among multiple application processes.
[0083] In this scheme, by carrying the connection identifier of the first destination in the fourth information, the second processor can determine which application process has completed data reception based on the connection identifier of the first destination.
[0084] In one possible implementation, the second network interface card (NIC) can also receive a second message, which includes a second payload and second header information. The second header information includes: a connection identifier for the second source end, a connection identifier for the second destination end, and an identifier for the second data. The second payload carries the second data, which corresponds to a different application process than the first data. Of course, the reception of the second message does not necessarily require QP. The second NIC can also report fifth information to the second computer device via CQ, indicating the completion of second data reception.
[0085] Similarly, the second payload can carry the entire second data or a portion of the second data. Furthermore, the connection identifier at the second source end and the connection identifier at the second destination end jointly identify another transmission channel.
[0086] Furthermore, the second data corresponds to different application processes than the first data. This can be understood as follows: the first data is the data that the third application process on the second computer device needs to receive, and the second data is the data that the fourth application process on the second computer device needs to receive. Alternatively, it can be understood as follows: the first data is the data transmitted between the first application process on the first computer device and the third application process on the second computer device, and the second data is the data transmitted between the second application process on the first computer device and the fourth application process on the second computer device.
[0087] In this scheme, on the one hand, the source connection identifier, destination connection identifier, and message sequence number of the payload data are carried in the header information of the data packet. Furthermore, the source and destination connection identifiers jointly identify a transmission channel, meaning that the source and destination connection identifiers in the data packet can be used together as the connection carrier, without requiring a specific QP as the connection carrier. This can also be understood as the data packet carrying transmission channel information, eliminating the need to establish a specific QP connection before sending the message; that is, the transmission channel information is unbound from a specific QP. This approach not only allows for flexible data packet transmission but also saves memory resources required to establish a specific QP. On the other hand, after the second network card reports the first data reception completion information via CQ, it can also report the second data reception completion information via CQ. That is, the CQ does not belong to a specific application process; different application processes can share a single CQ to transmit data completion information. This approach not only allows for flexible transmission of data completion information but also reduces memory resources by sharing the CQ.
[0088] In one possible implementation, the fifth piece of information may also include the connection identifier of the second destination.
[0089] In this scheme, by carrying the connection identifier of the second destination in the fifth information, the second processor can determine which application process has completed data reception based on the connection identifier of the second destination.
[0090] In one possible implementation, the aforementioned first data corresponds to multiple messages, and the first header information also includes an offset, which is used to indicate the position of the first message among the multiple messages. Alternatively, it can be understood that there are multiple first payloads, that is, the first data is carried by multiple first payloads, and the offset can also be understood as indicating the position of the first payload within the first data.
[0091] In this solution, the second network card can correctly splice multiple payloads using the offset in the first header information.
[0092] In one possible implementation, before storing the first data in the second memory, the second network interface card (NIC) may also receive a mapping table sent by the second processor. The mapping table records the mapping relationship between the connection identifier of the first destination and the target storage address, where the target storage address indicates the target storage space in the memory of the second computer device. The second NIC storing the first data in the second memory specifically includes: the second NIC storing the first data in the target storage space based on the mapping table. It is understood that the target memory address can be used for reading or writing. For example, "writing" corresponds to storing the first data carrying the first payload in the storage space corresponding to the target storage. Another example is "reading," which corresponds to copying the data at the target storage address to the temporary storage of the second NIC, thereby transmitting it to the first NIC.
[0093] The timing of the second network card obtaining the mapping table can be either before or after receiving the first packet; there is no limitation on the timing of the second network card obtaining the mapping table here.
[0094] In this solution, the second network interface card (NIC) can use a mapping table issued by the second processor to specify which part of the second processor's storage space to store the data. The second processor can then use the data stored by the second NIC in the corresponding storage space using the mapping table.
[0095] This application provides a data communication method in its fourth aspect. The method is executed by a second computer device, or by a component of the second computer device (e.g., a second processor, chip, or chip system), or by a logic module or software capable of implementing all or part of the functions of the second computer device. The second computer device communicates based on the Remote Direct Memory Access (RDMA) protocol, or, as understood, different computer devices communicate via RDMA protocol through network interface cards (NICs) connected to their respective processors. For example, the second computer device can communicate via RDMA with the first NIC of a first computer device through a second NIC. The second NIC can be plugged into and connected to the second processor, integrated into a chip, or connected via a bus, etc. The second NIC can also be called an RDMA NIC. In this fourth aspect and its possible implementations, the method is described using the example of execution by a second processor.
[0096] In this method, the second processor allocates a first destination connection identifier to the third application process and sends the first destination connection identifier to the first processor. The first destination connection identifier is used to represent the transmission channel of the first data. Alternatively, it can be understood that the first destination connection identifier is used to obtain the first data from the first network interface card.
[0097] In this scheme, multiple application processes on the second computer device can request a destination connection identifier, allowing the second processor to send this identifier to the first processor. This destination connection identifier represents the data transmission channel between network interface cards (NICs), enabling the second processor to obtain data from the sending NIC based on the destination connection identifier. Furthermore, since the destination connection identifier represents the first data transmission channel, the two processors do not need to establish a connection to a specific QP before sending messages; that is, the transmission channel information is unbound from the specific QP.
[0098] In one possible implementation, the first data is carried by a first payload, and the first header information of the first payload includes a connection identifier of the first destination end. The connection identifier of the first destination end is also used by the second network card to determine the destination storage address in the second memory, which is used to store the first data.
[0099] In this scheme, by including a connection identifier representing the destination of the transmission channel in the header information of the data packet, the connection identifiers of the source and destination ends in the data packet can be used together as the connection carrier, without requiring a specific QP as the connection carrier. This can also be understood as the data packet carrying transmission channel information, eliminating the need for the processors on both sides to establish a specific QP connection before sending the message; that is, the transmission channel information is decoupled from a specific QP. This approach not only allows for flexible data packet transmission but also saves the memory resources required to establish a specific QP. Furthermore, the second processor can determine which application process is receiving the data based on the destination connection identifier.
[0100] In one possible implementation, the second processor also receives a fourth message via CQ, which indicates that the first data reception is complete. CQ is used to transmit data reception completion information for multiple application processes on the second computer device.
[0101] In this scheme, after the second network interface card (NIC) receives the first data, it can send a fourth message to the second processor where the NIC resides. This allows the second processor to confirm that the first data transmission is complete, enabling related application processes to know that the first data has been correctly received and to proceed with subsequent operations. Furthermore, the second NIC sends the fourth message to the second processor through a CQ shared by multiple application processes. Compared to the method where the second NIC sends a CQE to the second processor through a specific QP, this reduces the maintenance of specific QPs.
[0102] In one possible implementation, the aforementioned fourth information includes a connection identifier of the first destination, which is used to indicate the third application process. Alternatively, it can be understood that the connection identifier of the first destination is used to clarify that the first data is related to the third application process among multiple application processes.
[0103] In this scheme, by carrying the connection identifier of the first destination in the fourth information, the second processor can determine which application process has completed data reception based on the connection identifier of the first destination.
[0104] In one possible implementation, the second processor may further allocate a connection identifier for the second destination to the fourth application process and send the connection identifier to the first processor. The connection identifier for the second destination is used to indicate the transmission channel for the second data. Alternatively, it can be understood that the connection identifier for the second destination is used to acquire the second data from the first network interface card. The second processor also receives a fifth message via CQ, which indicates that the second data reception is complete.
[0105] On the one hand, by identifying the transmission channel through the connection identifier at the destination, the connection identifiers at both the source and destination ends in the data packet can be used together as the connection carrier, eliminating the need for a specific QP. This can also be understood as the data packet carrying transmission channel information without needing to establish a specific QP connection before sending the message; that is, the transmission channel information is unbound from a specific QP. This approach not only allows for flexible data packet transmission but also saves memory resources required to establish a specific QP. On the other hand, after the second processor receives the reception completion information of the first data via CQ, it can also receive the reception completion information of the second data via CQ. This means that the CQ does not belong to a specific application process; different application processes can share a single CQ to transmit data completion information. This approach not only allows for flexible transmission of data completion information but also reduces memory resources by sharing the CQ.
[0106] In one possible implementation, the fifth piece of information may also include the connection identifier of the second destination.
[0107] In this scheme, by carrying the connection identifier of the second destination in the fifth information, the second processor can determine which application process has completed data reception based on the connection identifier of the second destination.
[0108] In one possible implementation, the second processor can also establish a mapping table that records the mapping relationship between the connection identifier of the first destination and the target storage address. The connection identifier of the first destination is associated with a third application process among multiple application processes. The connection identifier of the first destination is used to store the first data, and the target storage address is used to indicate the target storage space of the memory in the second computer device. After the second processor establishes the mapping table, it can send the mapping table to the second network card. The mapping table is used by the second network card to store the first data using the target storage space.
[0109] In this scheme, the second processor can send a mapping table to the second network card, so that the second network card can specify which part of the memory space in the second computer device to store the data. The second processor can then use the data stored by the second network card in the corresponding memory space through the sent mapping table.
[0110] The fifth aspect of this application provides a computer device. This computer device includes a first network interface card (NIC) and a first processor. The computer device communicates based on the Remote Direct Memory Access (RDMA) protocol; or, as can be understood, the computer device includes a first processor and a first NIC, the first NIC also being referred to as an RDMA NIC. The functions of each part in this computer device are as follows:
[0111] A first processor is configured to send first information to a first network interface card, the first information including: the storage address of first data and the length of the first data;
[0112] The first network interface card (NIC) is used to send a first message to the second NIC based on the first queue pair (QP). The first message includes a first payload and first header information. The first header information includes: a connection identifier of the first source end, a connection identifier of the first destination end, and an identifier of the first data. The first payload is used to carry the first data. In addition, the state of the first QP is idle.
[0113] In this scheme, the first processor can send a data packet corresponding to the first data through an idle first QP. The data packet header includes the source connection identifier, the destination connection identifier, and the message sequence number of the payload data. Furthermore, the source and destination connection identifiers jointly identify a transmission channel, meaning the source and destination connection identifiers in the data packet can serve as the connection carrier, eliminating the need for a specific QP. This can also be understood as the data packet carrying transmission channel information, without needing to establish a connection to a specific QP before sending the message; that is, the transmission channel information is unbound from a specific QP. This approach not only allows for flexible data packet transmission but also saves memory resources required to establish a specific QP. Moreover, since the transmission channel information is unbound from a specific QP, this method can also be called a message-based short connection compared to existing long connections with fixed QPs. For example, after application process 1 transmits a message using QP1, QP1 is set to idle, allowing application process 2 to also use QP1 to transmit messages, thus enabling multiple application processes to share a single QP.
[0114] In one possible implementation, the first processor is further configured to determine an idle first QP in the first queue pair group (QPG) and change the state of the first QP from idle to non-idle. The first information also includes first indication information, which is used to indicate that first data should be sent through the first QP.
[0115] In this scheme, the first processor can determine an idle first QP in the first QPG. It can indicate the idle first QP to the first network interface card (NIC) via first indication information, so that the first NIC can transmit first data through the idle first QP. Furthermore, by adjusting the state of the first QP, other application processes cannot use the first QP that is not in an idle state. Of course, if the state of the first QP is subsequently adjusted to idle, other application processes can use that idle QP to transmit data.
[0116] In one possible implementation, the first processor is further configured to change the state of the first QP from idle to non-idle after determining the first QP.
[0117] In this scheme, after determining the first QP, the first processor promptly adjusts the state of the first QP to non-idle. This state adjustment prevents other application processes from using the non-idle first QP. Of course, if the state of the first QP is subsequently adjusted to idle, other application processes can use that idle QP to transmit data.
[0118] In one possible implementation, the first QP includes: a SQ and a CQ; the SQ carries first information; a first network interface card (NIC) is specifically used to acquire the first data based on the SQ; the first NIC is specifically used to encapsulate the first data to obtain the first packet; and the first NIC is specifically used to send the first packet to a second NIC. Furthermore, the first NIC is also used to send second information to a first processor based on the CQ, the second information indicating that the first data transmission is complete. Correspondingly, the first processor receives the second information sent by the first NIC. For example, this second information could refer to a CQE.
[0119] In this scheme, after the first network card completes the transmission of the first data, it can send the second information to the first processor through CQ so that the first processor knows that the first data has been transmitted successfully, so that the first application process of the first data knows that the first data has been sent correctly and can perform subsequent operations.
[0120] In one possible implementation, the second information mentioned above includes a connection identifier of the first source end, which is associated with the first application process.
[0121] In this scheme, the first processor can determine that the data transmission of the first application process is complete by the connection identifier of the first source end.
[0122] In one possible implementation, the first processor or first network interface card (NIC) is further configured to change the state of the first QP from non-idle to idle; the first processor is further configured to send third information to the first NIC, the third information including: the storage address of the second data, the length of the second data, wherein the first data and the second data are generated by different application processes; the first NIC is further configured to send a second message to the second NIC based on the first QP, the second message including a second payload and second header information, the second header information including: a connection identifier of the second source end, a connection identifier of the second destination end, and an identifier of the second data, the second payload being used to carry the second data. The SQ in the first QP is used to carry the third information.
[0123] In this scheme, on the one hand, after message transmission is completed, the first processor or the first network card can change the state of the first QP from non-idle to idle, thus facilitating the subsequent transmission of other messages through the idle first QP. It can be seen that in this method, the first QP is promptly released for other message transmission after message transmission, thereby achieving message-based short connections. On the other hand, after the first application process's first data is sent through the first QP, the second data of the first application process / second application process can also be sent through the first QP. That is, the first QP does not necessarily belong to a specific application process; different application processes on the first computer device can share a single QP to send data, i.e., unbinding the transmission channel information from a specific QP, thereby enabling multiple application processes to share a single QP. Through the above method, not only can data transmission be flexibly achieved, but the memory resources required to establish a specific QP can also be saved.
[0124] In one possible implementation, the first QP belongs to a first queue pair group (QPG), which is used for transferring data between multiple application processes on a computer device based on RDMA. The first processor is also used to create the first QPG.
[0125] In this scheme, multiple application processes on a computer device can transmit data through a first QPG created by a first processor. For example, the first QPG can send data to application process 1 as well as application process 2. This allows multiple application processes to share the same QPG. This approach not only enables flexible data transmission but also saves the memory resources required to create a specific QPG for each application process.
[0126] In one possible implementation, the first network interface card (NIC) is further used to establish a first context for the first data, which maintains the transmission status of the first data. For example, the first context for the first data includes at least one of the following: the amount of data already transmitted, and the amount of data yet to be transmitted. It is understood that the first context may also include an offset, which, when the first data is transmitted through multiple packets, indicates the position of the transmitted first packet within the multiple packets. Of course, if a first preset condition is met, the first NIC can also be used to delete the first context; the first preset condition includes at least one of the following: all packets containing the first data have been transmitted; or, acknowledgment messages for all packets corresponding to the first data have been received from the second NIC. It is understood that, when the first data is transmitted through multiple packets, the first preset condition may be receiving an acknowledgment message for the last packet among the multiple packets.
[0127] In this scheme, the first network interface card (NIC) maintains the transmission status of the first data through an established first context, ensuring that the first data is transmitted correctly and received completely by the second NIC. For example, in the event of a lost packet, the first context can be used to determine which packets need to be retransmitted, guaranteeing that the second NIC reliably receives the first data. Furthermore, promptly deleting the data context after data transmission is complete not only saves storage space but also reduces the power consumption of the first NIC.
[0128] In one possible implementation, the first network interface card (NIC) is also used to receive an acknowledgment message corresponding to the first message from the second NIC. The acknowledgment message indicates that the second NIC has correctly received the first message. It is understood that, in order for the first NIC to clearly identify which message corresponds to the acknowledgment message, the acknowledgment message may also carry the three identifiers from the aforementioned first header information corresponding to the first message.
[0129] In this scheme, the first network interface card (NIC) can confirm that the second NIC has correctly received the first packet through the acknowledgment message from the second NIC. This facilitates other operations or subsequent operations for the first NIC and reduces timeout retransmissions. For example, after receiving an acknowledgment message for a packet, the first NIC can send subsequent packets to the second NIC to improve the transmission efficiency of the first data. As another example, after receiving an acknowledgment message for the last packet, the first NIC can delete the first context, saving storage space and reducing the power consumption of the first NIC.
[0130] In one possible implementation, the first information mentioned above further includes: a connection identifier of the first destination, which is used by the first network interface card to send the first data.
[0131] In this scheme, the first processor of the first computer device and the second processor of the second computer device can interact with the connection identifier of the first destination. Thus, the first processor can indicate the connection identifier of the first destination through the first information, which facilitates the first network card to transmit the first data. At the same time, it also makes it convenient for the second network card to determine which application process on the second computer device needs to receive the data after receiving the first message of the first data according to the connection identifier of the first destination.
[0132] In one possible implementation, the first information further includes: a connection identifier of a first source end, which is used by the first network interface card to send the first data; the first processor may also allocate a connection identifier of the first source end to the transmission channel, which, together with the connection identifier of the first destination end, identifies the transmission channel, and the transmission channel is used to transmit the first data.
[0133] In this scheme, the first processor can allocate a first source connection identifier for the transmission channel. This not only facilitates the joint identification of a transmission channel with the first destination connection identifier, but also allows the source and destination connection identifiers in the data packet to be used together as the connection carrier, eliminating the need for a specific QP as the connection carrier. In other words, the data packet can carry transmission channel information without needing to establish a specific QP connection before sending the message; that is, the transmission channel information is unbound from a specific QP. This approach not only enables flexible data packet transmission but also saves the memory resources required to establish a specific QP.
[0134] In one possible implementation, the first processor may further determine the available memory address in the memory of the second computer device; the first information also includes the available memory address, the storage space indicated by the available memory address being used to store the first data. For example, the first processor interacts with the second processor to determine the available memory address in the memory of the second computer device. The first processor can indicate the available memory address in the memory of the second computer device to the first network interface card (NIC) through the first information, so that the first NIC can carry the available memory address of the second computer device in the first header information, so that after receiving the first message, the second NIC can store the first data carried in the first payload according to the address carried in the first header information.
[0135] In this solution, this method can also be understood as a one-sided scenario. After receiving the first message, the second network card can determine which memory in the second computer device to store the first data carried by the first payload based on the available memory address carried by the first message.
[0136] In one possible implementation, the aforementioned first information may also include an identifier for the first data. Similarly, the first data may be identified solely by MSN, or it may be jointly identified by MSN and QPGN. Furthermore, the first information may also include a connection identifier for the first source end and a connection identifier for the first destination end.
[0137] In this scheme, the first processor can assign an identifier to the first data, and the first network card can clearly identify the first data through the first information, thereby facilitating the subsequent transmission of the first data and the establishment of the first context.
[0138] The sixth aspect of this application provides a computer device. This computer device includes a second network interface card (NIC) and a memory. The computer device communicates based on the Remote Direct Memory Access (RDMA) protocol, and the second NIC can also be referred to as an RDMA NIC. The functions of each part in this computer device are as follows:
[0139] The second network interface card (NIC) is used to receive the first message from the first NIC. The first message includes a first payload and first header information. The first header information includes: a connection identifier of the first source end, a connection identifier of the first destination end, and an identifier of the first data. The first payload is used to carry the first data. The reception of the first message does not use queue pairs (QP).
[0140] The memory is used to store the first data received by the second network card.
[0141] In this scheme, the header information of the data packets received by the second network interface card (NIC) carries the connection identifier of the source end, the connection identifier of the destination end, and the message sequence number of the payload data. Furthermore, the connection identifiers of the source end and the destination end jointly identify a transmission channel. This means that the connection identifiers of the source end and the destination end in the data packet can be used together as the connection carrier, without requiring a specific QP as the connection carrier. In other words, the data packet can carry the transmission channel information without establishing a specific QP connection before sending the message; that is, the transmission channel information is unbound from a specific QP. For example, the second NIC can receive data packets without using a QP. This method not only allows for flexible data packet transmission but also saves the memory resources required to establish a specific QP. Moreover, since the reception of data packets is independent of the QP, compared to existing long connections with fixed QPs, this method can also be called a data-based short connection.
[0142] In one possible implementation, the computer device further includes a second processor, a second network interface card, and is also used to send fourth information to the second processor, the fourth information being used to indicate that the first data transmission is complete.
[0143] In this scheme, after the second network card receives the first data, it can send a fourth message to the second processor so that the second processor knows that the first data has been successfully transmitted, so that the application processes related to the first data know that the first data has been correctly received and can perform subsequent operations.
[0144] In one possible implementation, the second network interface card (NIC) is specifically used to send a fourth message to the second processor via a completion queue (CQ). The CQ is used to transmit data reception completion information for multiple application processes on the second computer device. For example, this fourth message could refer to a CQE (Content Query Entrance).
[0145] After receiving the first data, the second network interface card (NIC) can send a fourth message to the second processor to confirm that the first data transmission is complete. This allows the application processes associated with the first data to know that it has been correctly received and proceed with subsequent operations. Furthermore, the second NIC sends the fourth message to the second processor through a CQ shared by multiple application processes. Compared to sending CQE to the second processor through a specific QP, this reduces the maintenance required for specific QPs.
[0146] The second network interface card is also used to receive a second message from the first network interface card. The second message includes a second payload and second information. The second header information includes: a connection identifier of the second source end, a connection identifier of the second destination end, and an identifier of the second data. The second payload is used to carry the second data. The first data and the second data correspond to different application processes on the computer device (for example, the first data corresponds to the third application process, and the second data corresponds to the fourth application process).
[0147] The memory is also used to store the second data received by the second network card.
[0148] In this scheme, the source connection identifier, destination connection identifier, and message sequence number of the payload data are included in the header information of the data packet. Furthermore, the source and destination connection identifiers jointly identify a transmission channel. This means that the source and destination connection identifiers in the data packet can serve as the connection carrier, eliminating the need for a specific QP as the connection carrier. In other words, the data packet can carry transmission channel information without needing to establish a specific QP connection before sending the message; that is, the transmission channel information is unbound from a specific QP. This approach not only allows for flexible data packet transmission but also saves the memory resources required to establish a specific QP.
[0149] In one possible implementation, the second network interface card is also used to report fifth information to the second computer device via CQ, the fifth information being used to indicate that the reception of the second data is complete.
[0150] In this solution, after the second network card reports the first data reception completion information via CQ, it can also report the second data reception completion information via CQ. That is, the CQ does not belong to a specific application process; different application processes can share a single CQ to transmit data completion information. This approach not only allows for flexible transmission of data completion information but also reduces memory resources by sharing the CQ.
[0151] In one possible implementation, the second network interface card (NIC) is further used to establish a second context for the first data. This second context is used to maintain the reception status of the first data. For example, the second context for the first data includes at least one of the following: the amount of first data already received, and the amount of data yet to be received. It is understood that the second context may also include an offset, which, in the case of the first data being transmitted through multiple packets, indicates the position of the transmitted first packet within the multiple packets. Of course, after the second NIC establishes the second context for the first data, it can also delete the second context if a second preset condition is met. The second preset condition includes at least one of the following: acknowledgment messages for all packets corresponding to the first data have been sent; or, fourth information has been sent to the second processor; the fourth information indicates that the reception of the first data is complete.
[0152] In this scheme, the second network interface card (NIC) maintains the reception status of the first data through an established second context, ensuring that the first data is correctly transmitted and completely received by the second NIC. For example, in the event of a lost packet, the second context can be used to determine which packets need to be retransmitted, guaranteeing reliable reception of the first data by the second NIC. Furthermore, promptly deleting the data context after data transmission not only saves storage space but also reduces the power consumption of the second NIC.
[0153] In one possible implementation, the second network interface card (NIC) is also used to send an acknowledgment message for the first message to the first NIC. The acknowledgment message indicates that the first message has been correctly received. Understandably, to allow the first NIC to identify which message corresponds to the acknowledgment message, the acknowledgment message may also carry the three identifiers from the aforementioned first header information corresponding to the first message.
[0154] In this scheme, the second network interface card (NIC) can send an acknowledgment message to the first NIC, confirming that the second NIC has correctly received the first packet. This facilitates other operations or subsequent operations for the first NIC and reduces timeout retransmissions. For example, after receiving an acknowledgment message for a packet, the first NIC can send subsequent packets to the second NIC to improve the transmission efficiency of the first data. As another example, after sending an acknowledgment message for the last packet, the second NIC can delete the second context to reduce the power consumption incurred by the second NIC in maintaining the second context.
[0155] In one possible implementation, the fourth piece of information mentioned above includes a connection identifier for the first destination. The connection identifier for the first destination is used to clarify that the first data is associated with a third application process among multiple application processes.
[0156] In this scheme, by carrying the connection identifier of the first destination in the fourth information, the second processor can determine which application process has completed data reception based on the connection identifier of the first destination.
[0157] In one possible implementation, the fifth piece of information mentioned above includes a connection identifier for the second destination. The connection identifier for the second destination is used to clarify that the second data is associated with a fourth application process among multiple application processes.
[0158] In this scheme, by carrying the connection identifier of the second destination in the fifth information, the second processor can determine which application process has completed data reception based on the connection identifier of the second destination.
[0159] In one possible implementation, the aforementioned first data corresponds to multiple messages, and the first header information also includes an offset, which is used to indicate the position of the first message among the multiple messages. Alternatively, it can be understood that there are multiple first payloads, that is, the first data is carried by multiple first payloads, and the offset can also be understood as indicating the position of the first payload within the first data.
[0160] In this solution, the second network card can correctly splice multiple payloads using the offset in the first header information.
[0161] In one possible implementation, the second processor may also assign a connection identifier for the first destination end to the transmission channel. The connection identifier for the first destination end and the connection identifier for the first source end jointly identify the transmission channel, which is a channel for transmitting the first data.
[0162] In this scheme, the second processor can assign a connection identifier for the first destination end of the transmission channel, which can be used together with the connection identifier for the first source end to identify a transmission channel. This can also be understood as data packets carrying transmission channel information without needing to establish a connection to a specific QP before receiving messages; that is, the transmission channel information is unbound from a specific QP. This approach not only allows for flexible data packet transmission but also saves the memory resources required to establish a specific QP.
[0163] The seventh aspect of this application provides a first network interface card (NIC) that supports the Remote Direct Memory Access (RDMA) protocol. The first NIC can also be referred to as an RDMA NIC, and it is located in a first computer device containing a first processor. The first NIC includes a transceiver unit and a processing unit, the functions of which are as follows:
[0164] The transceiver unit is used to receive first information sent by the first processor. The first information includes the storage address of the first data and the length of the first data.
[0165] The processing unit is configured to send a first message to a second network interface card (NIC) in a second computer device based on a first queue (QP). The first message includes a first payload and a first header. The first header includes a connection identifier of a first source end, a connection identifier of a first destination end, and an identifier for first data. The first payload carries the first data. The identifier for the first data uniquely identifies the first data. Furthermore, the first QP is in an idle state.
[0166] In one possible implementation, the processing unit is further configured to determine the first idle QP in the first queue pair group (QPG). The processing unit is also configured to change the state of the first QP from idle to non-idle.
[0167] In one possible implementation, the first information further includes first indication information, which is used to indicate that the first data is sent through the first QP.
[0168] In one possible implementation, the processing unit is further configured to establish a first context for the first data, the first context being used to maintain the transmission status of the first data. The processing unit is also configured to delete the first context if a first preset condition is met; the first preset condition includes at least one of the following: all packets of the first data have been transmitted; or, acknowledgment messages have been received from the second network interface card for all packets corresponding to the first data.
[0169] In one possible implementation, the transceiver unit is further configured to receive a response message corresponding to the first message from the second network interface card (NIC), the response message indicating that the second NIC has correctly received the first message.
[0170] In one possible implementation, the first QP includes: a transmit queue SQ and a complete queue CQ; the SQ carries first information; a processing unit is specifically used to acquire the first data based on the SQ; a processing unit is specifically used to encapsulate the first data with a first network interface card (NIC) to obtain the first packet; and a processing unit is specifically used to send the first packet to a second NIC. Furthermore, the transceiver unit is also used to send second information to a first processor based on the CQ in the first QP, the second information indicating that the first data transmission is complete.
[0171] In one possible implementation, the second information mentioned above includes the connection identifier of the first source end.
[0172] In one possible implementation, the processing unit is also used to change the state of the first QP from non-idle to idle.
[0173] In one possible implementation, the transceiver unit is also used to receive third information sent by the first processor. The third information includes: the storage address of the second data, the length of the second data, and the first data and the second data are generated by different application processes.
[0174] The processing unit is also configured to send a second message based on the first QP. The second message includes a second payload and second header information. The second payload is used to carry second data. The second header information includes: a connection identifier of the second source end, a connection identifier of the second destination end, and an identifier of the second data.
[0175] In one possible implementation, the aforementioned first data corresponds to multiple messages, and the first header information also includes an offset, which is used to indicate the position of the transmitted first message among the multiple messages. Alternatively, it can be understood that there are multiple first payloads, that is, the first data is carried by multiple first payloads, and the offset can also be understood as indicating the position of the first payload within the first data.
[0176] In one possible implementation, the aforementioned first information may also include an identifier for the first data. Similarly, the first data may be identified solely by MSN, or it may be jointly identified by MSN and QPGN. Furthermore, the first information may also include a connection identifier for the first source end and a connection identifier for the first destination end.
[0177] In one possible implementation, the first QP mentioned above belongs to a queue pair group first QPG, which is created by a first processor and is used to transfer data of multiple application processes on a first computer device based on RDMA.
[0178] In one possible implementation, the aforementioned first QPG is a QPG shared by multiple application processes, including a first application process and a second application process. The first data is generated by the first application process, and the second data is generated by the second application process.
[0179] In one possible implementation, the processing unit is further configured to determine the identifier of the first data, which is used by the first network interface card to send the first message.
[0180] In one possible implementation, the first information and the first header information also include the available memory address in the second memory where the second network card is located, and the available memory address is used to store the first payload.
[0181] This application provides an eighth aspect of a first computer device, which includes a first processor and a first network interface card (NIC). The first computer device communicates based on the Remote Direct Memory Access (RDMA) protocol, or, more broadly, different computer devices communicate via their respective NICs connected to them using the RDMA protocol. For example, the first computer device can communicate via RDMA with a second network interface card (NIC) of a second computer device. The first NIC can be plugged into and connected to the first processor, or it can be connected via a chip. The first NIC can also be referred to as an RDMA NIC. The first processor includes a transceiver unit and a processing unit, the functions of which are as follows:
[0182] The processing unit is used to create a first queue pair group (QPG), which includes multiple queue pairs (QPs). The first QPG is used for transmitting data from multiple application processes via RDMA. This can also be understood as the first QPG being shared by multiple application processes on a first computer device. Furthermore, the creation of the first QPG by the processing unit can be completed in one step or in multiple steps. For example, the processing unit first creates the first QPG, which has a small number of QPs, and then subsequently creates new QPs for the first QPG based on data transmission needs or application process requests.
[0183] The transceiver unit is used to acquire the first data of the first application process among multiple application processes.
[0184] The transceiver unit is also used to send first information to the first network interface card (NIC), the first information including the storage address of first data and the length of the first data. The first data can refer to any data (or any message) that a first application process in the first computer device wants to send.
[0185] In one possible implementation, the processing unit is further configured to determine a first idle QP among multiple QPs. The first information also includes first indication information, which instructs the first network interface card (NIC) to send first data through the first QP. After determining the first QP, the state of the first QP is changed from idle to non-idle. The first QP includes a transmit queue (SQ) and a complete queue (CQ); the SQ is used to carry the first information.
[0186] In one possible implementation, the transceiver unit is further configured to receive second information sent by the first network interface card based on the CQ in the first QP, the second information indicating that the first data transmission is complete. For example, the second information could refer to CQE.
[0187] In one possible implementation, the second information mentioned above includes a connection identifier of the first source end, which is associated with the first application process.
[0188] In one possible implementation, the processing unit is also used to change the state of the first QP from non-idle to idle.
[0189] In one possible implementation, the transceiver unit is further configured to acquire second data from a second application process among multiple application processes; the transceiver unit is also configured to send third information to the first network interface card (NIC), the third information including: the storage address of the second data and the length of the second data, the third information being used by the first NIC to send the second data through the first QP. The SQ in the first QP is also used to carry the third information.
[0190] In one possible implementation, the first information mentioned above further includes: a connection identifier of the first destination, which is used by the first network card to send the first data; the first processor may also receive the connection identifier of the first destination sent by the second processor.
[0191] In one possible implementation, the first information further includes: a connection identifier of a first source end, which is used by the first network interface card to send the first data; the first processor may also allocate a connection identifier of the first source end to the transmission channel, which, together with the connection identifier of the first destination end, identifies the transmission channel, and the transmission channel is used to transmit the first data.
[0192] In one possible implementation, the processing unit is further configured to determine the available storage address of the memory in the second computer device; the first information also includes the available storage address, the storage space indicated by the available storage address being used to store the first data.
[0193] In one possible implementation, the aforementioned first information may also include an identifier for the first data. Similarly, the first data may be identified solely by MSN, or it may be jointly identified by MSN and QPGN. Furthermore, the first information may also include a connection identifier for the first source end and a connection identifier for the first destination end.
[0194] The ninth aspect of this application provides a second network interface card (NIC) that supports the Remote Direct Memory Access (RDMA) protocol. The second NIC can also be referred to as an RDMA NIC, and it is located in a second computer device containing a second memory. The second NIC includes a transceiver unit and a processing unit. The functions of each unit are as follows:
[0195] The transceiver unit is used to receive a first message from a first network interface card (NIC). The first message includes a first payload and a first header. The first header includes a connection identifier of a first source end, a connection identifier of a first destination end, and an identifier of first data. The first payload carries the first data. The processing unit is used to store the first data in a second memory. The reception of the first message does not use a queue pair (QP).
[0196] In one possible implementation, the processing unit is further configured to establish a second context for the first data, which is used to maintain the reception status of the first data. For example, the second context for the first data includes at least one of the following: the amount of first data that has been received, and how much data has not yet been received. Of course, if a second preset condition is met, the processing unit can also be configured to delete the second context. The second preset condition includes at least one of the following: acknowledgment messages for all messages corresponding to the first data have been sent; or, fourth information has been sent to the second processor; the fourth information is used to indicate that the reception of the first data is complete.
[0197] In one possible implementation, the transceiver unit is further configured to send an acknowledgment message for the first packet to the first network interface card (NIC). The acknowledgment message indicates that the first packet has been correctly received. It is understood that, in order for the first NIC to clearly identify which packet corresponds to the acknowledgment message, the acknowledgment message may also carry the three identifiers from the aforementioned first header information corresponding to the first packet.
[0198] In one possible implementation, the transceiver unit is further configured to send a fourth message to the second processor, indicating that the first data reception is complete. Similarly, this process can also be understood as the transmission of the fourth message between the second network interface card (NIC) and the second processor of the second computer device on which the NIC resides. For example, the transceiver unit may specifically be configured to send the fourth message to the second processor based on a CQ (Custom Queuing) mechanism. The CQ mechanism is used to transmit data reception completion information for multiple application processes on the second computer device. That is, the fourth message could refer to CQE (Custom Queuing Equipment).
[0199] In one possible implementation, the fourth piece of information mentioned above includes a connection identifier for the first destination. The connection identifier for the first destination is used to clarify that the first data is associated with a third application process among multiple application processes.
[0200] In one possible implementation, the aforementioned transceiver unit is specifically used to send fourth information to the second processor via an idle second QP, wherein the second QP is determined by the second processor or the second network card.
[0201] In one possible implementation, the transceiver unit is further configured to receive a second message, the second message including a second payload and second header information. The second header information includes: a connection identifier of a second source end, a connection identifier of a second destination end, and an identifier of the second data. The second payload carries the second data, which corresponds to a different application process than the first data. The reception of the second message may not require the use of QP.
[0202] In one possible implementation, the second network card can also send a fifth message to the second processor via CQ, the fifth message being used to indicate that the reception of the second data is complete.
[0203] In one possible implementation, the fifth piece of information may also include the connection identifier of the second destination.
[0204] In one possible implementation, the aforementioned first data corresponds to multiple messages, and the first header information also includes an offset, which is used to indicate the position of the first message among the multiple messages. Alternatively, it can be understood that there are multiple first payloads, that is, the first data is carried by multiple first payloads, and the offset can also be understood as indicating the position of the first payload within the first data.
[0205] In one possible implementation, the transceiver unit is further configured to receive a mapping table sent by the second processor. The mapping table records the mapping relationship between the connection identifier of the first destination and the target storage address, and the target storage address is used to indicate the target storage space of the memory in the second computer device. The processing unit is specifically configured to store the first data in the second memory, specifically including: the second network card storing the first data in the target storage space based on the mapping table.
[0206] The tenth aspect of this application provides a second computer device, which includes a second processor and a second network interface card (NIC). This second computer device communicates based on the Remote Direct Memory Access (RDMA) protocol, or, as can be understood, different computer devices communicate with each other via their respective NICs using the RDMA protocol. For example, the second computer device can communicate with the first NIC of a first computer device via the second NIC using RDMA. The second NIC can be connected to the second processor in a pluggable manner or via a chip, etc. The second NIC can also be referred to as an RDMA NIC. The second processor includes a transceiver unit and a processing unit, the functions of which are as follows:
[0207] The processing unit is used to allocate a connection identifier for a first destination to a third application process. The connection identifier for the first destination is used to indicate the transmission channel for the first data. Alternatively, it can be understood that the connection identifier for the first destination is used to obtain the first data from the first network interface card.
[0208] The transceiver unit is used to send the connection identifier of the first destination to the first processor.
[0209] In one possible implementation, the first data is carried by a first payload, and the first header information of the first payload includes a connection identifier of the first destination end. The connection identifier of the first destination end is also used by the second network card to determine the destination storage address in the second memory, which is used to store the first data.
[0210] In one possible implementation, the second processor also receives a fourth message via CQ, which indicates that the first data reception is complete. CQ is used to transmit data reception completion information for multiple application processes on the second computer device.
[0211] In one possible implementation, the aforementioned fourth information includes a connection identifier of the first destination, which is used to indicate the third application process. Alternatively, it can be understood that the connection identifier of the first destination is used to clarify that the first data is related to the third application process among multiple application processes.
[0212] In one possible implementation, the processing unit is further configured to allocate a connection identifier for the second destination end to the fourth application process. This connection identifier represents the transmission channel for the second data. Alternatively, the connection identifier for the second destination end can be understood as being used to acquire the second data from the first network interface card.
[0213] The transceiver unit is also used to send the connection identifier of the second destination to the first processor.
[0214] In one possible implementation, the transceiver unit is also configured to receive a fifth message via CQ, the fifth message being used to indicate that the second data reception is complete.
[0215] In one possible implementation, the fifth piece of information mentioned above includes a connection identifier for the second destination, which is used to indicate the fourth application process. Alternatively, it can be understood that the connection identifier for the second destination is used to clarify that the second data is associated with the fourth application process among multiple application processes.
[0216] In one possible implementation, the processing unit is further configured to establish a mapping table, which records the mapping relationship between the connection identifier of the first destination and the target storage address. The connection identifier of the first destination is associated with a third application process among multiple application processes. The connection identifier of the first destination is used to store first data, and the target storage address is used to indicate the target storage space of the memory in the second computer device. The transceiver unit is further configured to send the mapping table to the second network card, which is used by the second network card to store the first data using the target storage space.
[0217] The eleventh aspect of this application provides a communication system comprising: a computer device as described in any possible implementation of any of the fifth aspects above, and a computer device as described in any possible implementation of any of the sixth aspects above. Alternatively, it may include: a first network interface card (NIC) as described in any possible implementation of any of the seventh aspects above, and a first processor as described in any possible implementation of any of the eighth aspects above. Alternatively, it may include: a second NIC as described in any possible implementation of any of the ninth aspects above, and a second processor as described in any possible implementation of any of the tenth aspects above. Alternatively, it may include: a first NIC as described in any possible implementation of any of the seventh aspects above, a first processor as described in any possible implementation of any of the eighth aspects above, a second NIC as described in any possible implementation of any of the ninth aspects above, and a second processor as described in any possible implementation of any of the tenth aspects above.
[0218] The twelfth aspect of this application provides a network interface card (NIC) including a memory and a processor; the memory stores code, and the processor is configured to execute the code, wherein when the code is executed, the NIC performs a method as described in any possible implementation of any of the first aspects above; or the NIC performs a method as described in any possible implementation of any of the third aspects above.
[0219] In one possible implementation, the aforementioned network card is a chip or chip system.
[0220] The thirteenth aspect of this application provides a computer device, characterized in that the computer device includes a processor, a memory, and a network interface card (NIC), the NIC being communicatively connected to the processor and the memory, and the NIC being configured to perform, based on instructions from the processor, a method for executing, with respect to the memory, any possible implementation of any of the first aspects described above, or a method for executing, with respect to the memory, any possible implementation of any of the third aspects described above.
[0221] The fourteenth aspect of this application provides a computer-readable storage medium for storing one or more computer-executable instructions, which, when executed by a processor, perform the method as described in any possible implementation of any of the first to fourth aspects described above.
[0222] The fifteenth aspect of this application provides a computer program product (or computer program) that, when executed by a processor, performs the method described in any possible implementation of any of the first to fourth aspects described above.
[0223] A sixteenth aspect of this application provides a computing device cluster, including at least one computer device as described in the fifth or sixth aspect, wherein any one computer device is used to run a computer program or instructions, enabling the computing device cluster to perform the methods of any of the above aspects. Alternatively, some or all of the computer devices may be used together to run a computer program or instructions, enabling the computing device cluster to perform the methods of any of the above aspects.
[0224] The technical effects of any of the design methods in aspects seven through sixteen can be found in the technical effects of the different design methods in aspects one through six above, and will not be repeated here. Attached Figure Description
[0225] Figure 1 is a schematic diagram of the data communication system provided in this application;
[0226] Figure 2 is a schematic diagram of the software structure of the computer device provided in this application;
[0227] Figure 3 is a schematic diagram of the intelligent message scheduler provided in this application;
[0228] Figure 4 is a flowchart illustrating the data communication method provided in this application;
[0229] Figure 5 is a structural example of the first message provided in this application;
[0230] Figure 6 is another flowchart illustrating the data communication method provided in this application;
[0231] Figure 7 is another flowchart illustrating the data communication method provided in this application;
[0232] Figure 8 is a schematic diagram of a network card provided in an embodiment of this application;
[0233] Figure 9 is a structural schematic diagram of a computer device provided in an embodiment of this application;
[0234] Figure 10 is another structural schematic diagram of the computer device provided in an embodiment of this application;
[0235] Figure 11 is another structural schematic diagram of the computer device provided in an embodiment of this application. Detailed Implementation
[0236] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application are described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some, and not all, of the embodiments of this application. Those skilled in the art will understand that, with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0237] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such descriptions can be used interchangeably where appropriate to allow embodiments to be implemented in a sequence other than that illustrated or described in this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps appearing in this application does not imply that the steps in the method flow must be performed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical purpose, as long as the same or similar technical effect is achieved. The division of units in this application is a logical division. In practical applications, there may be other division methods. For example, multiple units may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the shown or discussed mutual coupling, direct coupling, or communication connection may be through some interface, and the indirect coupling or communication connection between units may be electrical or other similar forms, none of which are limited in this application. Furthermore, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed among multiple circuit units. Some or all of the units can be selected to achieve the purpose of the solution in this application according to actual needs.
[0238] RDMA is a high-efficiency network communication technology that allows computer systems to directly access the memory of remote computers over a network without the intervention of the central processing unit (CPU). The advantages of RDMA technology include reducing the CPU load, increasing data transfer speed and bandwidth, and reducing network latency.
[0239] Currently, RDMA technology often uses the RC (Redirect Memory Access) approach. Specifically, for each connection: First, the sending and receiving ends need to each determine a specific QP (Queued Message Block). Second, the sending and receiving ends establish a connection between the QPs (e.g., socket, CM, etc.). Finally, the message is transmitted from the sending queue of the sending-specific QP to the receiving queue of the receiving-specific QP, thus realizing message transmission.
[0240] However, the above method requires establishing a specific QP before transmitting messages, and messages can only be transmitted through that specific QP. In scenarios with large-scale access, the number of specific QPs can be enormous, consuming a huge amount of memory resources.
[0241] To address the aforementioned technical problems, this application provides a data communication method and related equipment. During interaction, two network interface cards (NICs) carry the connection identifier of the source end, the connection identifier of the destination end, and the message sequence number of the payload data in the header information of the data packet. Furthermore, the connection identifier of the source end and the connection identifier of the destination end jointly identify a transmission channel. This means that the connection identifiers of the source end and the destination end in the data packet can be used together as the connection carrier, without requiring a specific QP as the connection carrier. In other words, the data packet can carry the transmission channel information without establishing a specific QP connection before sending the message; that is, the transmission channel information is unbound from the specific QP. This method not only allows for flexible data packet transmission but also saves the memory resources required to establish a specific QP. Moreover, since the transmission channel information is unbound from the specific QP, compared to the existing long connections with fixed QPs, this method can also be called a message-based short connection. For example, after application process 1 uses QP1 to transmit a message, QP1 is set to idle, allowing application process 2 to also use QP1 to transmit messages, thus enabling multiple application processes to share a single QP.
[0242] Figure 1 is a schematic diagram of a data communication system provided in an embodiment of this application. As shown in Figure 1, the data communication system includes a first computer device 101 and a second computer device 102.
[0243] The first computer device 101 and the second computer device 102 can communicate based on a protocol that supports RDMA technology. This protocol can be any one of the following: infinite bandwidth (IB) protocol, RDMA over converged Ethernet (RoCE) protocol, and Internet wide area RDMA protocol (iWARP), etc.
[0244] The first computer device 101 includes a first network interface card (NIC) 1011 and a first processor 1012. The NIC 1011 and the processor 1012 are connected via a bus 1013. Optionally, the first computer device 101 may also include a first memory. Correspondingly, the second computer device 102 includes a second NIC 1021 and a second processor 1022. The NIC 1021 and the processor 1022 are connected via a bus 1023. Optionally, the second computer device 102 may also include a second memory. The first NIC 1011 and the second NIC 1021 may also be referred to as RDMA NICs or smart NICs, etc.
[0245] Alternatively, the first network interface card (NIC) 1011 or the second NIC 1021 may also include a processor. The processor in the NIC can also be referred to as an RDMA engine. The processor in the NIC is mainly used for network protocol processing and data transmission processing. Network protocol processing mainly includes at least one of the following: packet parsing, routing, error detection, etc. Data transmission processing mainly includes at least one of the following: packet encapsulation and decapsulation, data copying, etc. For example, the received data is stored in the aforementioned first or second memory (also referred to as the computer device's RAM).
[0246] The aforementioned bus 1013 or bus 1023 can be the Peripheral Component Interconnect Express (PCIe) standard, or it can be other types of buses that enable connections between devices within a device. Furthermore, in addition to the data bus, bus 1013 or bus 1023 can also include power buses, control buses, and status signal buses. Of course, the network card can also be connected to the processor through chip integration or other methods; specific details are not limited here.
[0247] Furthermore, communication between the first computer device 101 and the second computer device 102 can take many forms. For example, the first processor 1012 in the first computer device 101 communicates with the second processor 1022 in the second computer device 102. This communication method between the processors of the computer devices can include at least one of the following: out-of-band switching, CM (Communication Management), etc. Out-of-band switching can also be referred to as a socket. As another example, the first network interface card (NIC) 1011 in the first computer device 101 communicates with the second NIC 1021 in the second computer device 102. This communication between NICs can be referred to as communication based on the RDMA protocol. The transmission between the processors of the computer devices can also be referred to as control plane transmission, and the transmission between the NICs of the computer devices can also be referred to as data plane transmission.
[0248] The data communication method provided in this application relates to communication between computer devices (e.g., the first and second computer devices shown in Figure 1) or a cluster of computer devices. A detailed description will follow with reference to the accompanying drawings; here, the overall process is simply described and will not be elaborated further. For example, a computer device can be understood as a computer device including a host and a network interface card (NIC), wherein the host includes a processor (e.g., the first or second processor shown in Figure 1) and a memory. Accordingly, the memory can be referred to as the host's RAM.
[0249] Taking the two computer devices shown in Figure 1 as an example, the overall communication process includes: the first network interface card (NIC) 1011 in the first computer device 101 acquires the data to be transmitted, encapsulates the data to be transmitted into a data packet, and directly sends the data packet to the second NIC 1021 in the second computer device 102. After receiving the data packet sent by the first NIC 1011, the second NIC 1021 decapsulates the data packet and stores the corresponding data in the memory of the second computer device 102. That is, the data transmission does not require CPU participation; the data is directly copied from user space to the first NIC, then transmitted through the physical link to the remote second NIC, and then directly sent into user space. It can be understood that the available memory address in the second computer device 102 can be used for reading or writing. For example, "writing" corresponds to storing the first data carried by the first payload in the storage space corresponding to the available memory address in the memory of the second computer device 102. As another example, "reading" corresponds to copying the data at the available memory address in the memory of the second computer device 102 to the temporary storage of the second NIC 1021, thereby transmitting it to the first NIC 1011.
[0250] In this application, the computer equipment (e.g., the first computer equipment 101 and the second computer equipment 102) can be a cloud server, network server, application server, management server, client, mobile phone, in-vehicle media playback device, wearable electronic device, virtual reality (VR) terminal device, augmented reality (AR) device, vehicle, in-vehicle terminal, aircraft terminal, intelligent robot, storage device with processor (e.g., storage array), etc., or a server. Furthermore, the number of network cards, processors, or memory included in the computer equipment can be one or more, and this is not specifically limited here.
[0251] In this application, the processor may include any one or more of the following: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), data processing unit (DPU), neural processing unit (NPU), or digital signal processor (DSP).
[0252] In this application, the memory may include volatile memory, such as random access memory (RAM). The memory may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).
[0253] For example, the data communication system shown in Figure 1 can also be understood as a computing device cluster. A computing device cluster includes at least one computer device. The memory of one or more computer devices in the computing device cluster may store the same instructions for performing data communication methods.
[0254] In some possible implementations, the memories of one or more computer devices in the computing device cluster may also each store a portion of the instructions for executing the data communication method. In other words, a combination of one or more computer devices can jointly execute the instructions for executing the data communication method.
[0255] The above describes the data communication system from a purely hardware perspective. The following section describes the computer device shown in Figure 1 from a software perspective. Figure 2 is a schematic diagram of the software structure of the computer device provided in this application. It can be seen that this software structure includes: an application layer, an interface layer, a queue pair group (QPG) service layer, a driver layer, and a network interface card (NIC).
[0256] The application layer, acting as the upper-layer application, can call the interfaces of the interface layer to implement its own business logic and achieve data transmission. The interface layer provides a unified interface for the application layer to call its interfaces. The QPG service layer is mainly used for: 1. Scheduling idle QPs. 2. Creating and managing at least one QPG. The driver layer is mainly responsible for the processor driving the network interface card (NIC) to complete the corresponding functions. The NIC layer is mainly responsible for editing and parsing new packet headers, etc. The NIC layer can implement a reliable transmission mechanism at the message granularity level. Furthermore, the NIC layer can select idle QPs for data transmission. For the NIC layer acting as the receiving side, it can also perform data addressing, etc.
[0257] In this application, a computer device can create at least one QPG through the QPG service layer, and each QPG includes at least one QP. The QPG is used for transmitting data from multiple application processes on the computer device based on RDMA. This can also be understood as multiple application processes on the computer device sharing the QPG. Furthermore, the process of creating at least one QPG can be completed in one step or in multiple steps. For example, the computer device can create all the QPs in the QPG at once. Alternatively, the computer device can first create some of the QPs in the QPG, and then create new QPs for the QPG later based on data transmission needs or application process requests.
[0258] Each of the at least one QP mentioned above includes: a send queue (SQ), a receive queue (RQ), and a completion queue (CQ). The SQ is used to send data, the RQ is used to receive data, and the CQ is used to send completion messages. Elements in the SQ can also be called SQEs (send queue elements), elements in the RQ can also be called RQEs (receive queue elements), and elements in the CQ can also be called CQEs (completion queue elements). Accordingly, the aforementioned receiver-side network interface card (NIC) layer can implement data addressing through the RQE.
[0259] As shown in Figure 2, QPG can include QP1 and QP2. QP1 includes SQ1, RQ1, and CQ1. QP2 includes SQ2, RQ2, and CQ2.
[0260] It is understood that Figure 2 is only used as an example to illustrate the concept of QPG including QP1 and QP2. In actual applications, the QPG service layer can create at least one QPG, and each QPG includes at least one QP. The specific number of QPG, QP, SQ, RQ, CQ, etc. is not limited here.
[0261] Optionally, the QPG service layer can also assign connection identifiers to multiple application processes, which are used to represent the data transmission channel. For example, the QPG service layer on the sending side can assign a source connection identifier to the application process, and the receiving processor can assign a destination connection identifier to the application process (specifically, through the QPG service layer or other methods). The source connection identifier and the destination connection identifier can jointly and uniquely represent a transmission channel. Before transmitting data, the two processors can exchange the destination connection identifier. Thus, the sending side's QPG service layer can inform the sending side's network card of its assigned source connection identifier and the destination connection identifier assigned by the receiving side. Therefore, when encapsulating data, the sending side's network card can add the source connection identifier and the destination connection identifier to the header information corresponding to the data (i.e., the aforementioned new message header). Thus, the message sent to the receiving side's network card carries the source connection identifier and the destination connection identifier. Since the source connection identifier and the destination connection identifier jointly identify a transmission channel, the source connection identifier and the destination connection identifier in the data packet can be used together as the connection carrier, without requiring a specific QP as the connection carrier. This can also be understood as data packets carrying transmission channel information, eliminating the need to establish a connection to a specific QP before sending the message; that is, debinding the transmission channel information from a specific QP. This approach not only allows for flexible data packet transmission but also saves the memory resources required to establish a specific QP.
[0262] Furthermore, as the sending-side computer device, the QPG service layer within the computer device can also schedule at least one QP through the intelligent message scheduler shown in Figure 3. The intelligent message scheduler can schedule messages sent by application processes to the corresponding idle QP for transmission according to the user-specified scheduling policy. Alternatively, it can be understood that each message can be sent to the sending-side network card through any idle QP.
[0263] For example, as shown in Figure 3, the intelligent message scheduler determines that messages 1, 2, 3, and 4 need to be sent. The intelligent message scheduler selects the idle QP1 to send message 3 to the sending-side network card, selects the idle QP2 to send message 4 to the sending-side network card, and selects the idle QP3 to send messages 1 and 2 to the sending-side network card.
[0264] Optionally, scheduling strategies are provided as plugins or can be user-defined. Specific scheduling strategies can take various forms, such as round-robin scheduling, random scheduling, priority scheduling, and AI-powered intelligent scheduling. The QPG service layer proxies external communication between multiple or all application processes on the computer device. Additionally, the QPG service layer can also provide services externally in the form of a Connection Queue (CNQ). After an application process requests a connection queue, it uses this queue to send and receive messages with the network interface card.
[0265] Furthermore, the aforementioned data communication system can be applied to both one-sided and two-sided scenarios, without specific limitations here. The main difference between one-sided and two-sided scenarios lies in how the network interface card (NIC) on the receiving side determines which address in the computer device's memory to store the received data.
[0266] In a one-sided scenario, before data transmission, the processors of the computers hosting the sending and receiving network interface cards (NICs) exchange information about the available memory addresses in the receiving computer's memory. Therefore, when the sending NIC sends data to the receiving NIC, it can carry the available memory address of the receiving computer. After receiving the data, the receiving NIC stores the data in the corresponding available memory address in the receiving computer's memory. For example, in a one-sided scenario, for data to be "written," the receiving side informs the sending side in advance of the address of the writable data in its memory, and the receiving NIC writes the message to that address. For data to be "read," the receiving side informs the sending side in advance of the address of the readable data in its memory, and the receiving NIC copies the data from the corresponding address to its temporary storage before transmitting it to the first NIC.
[0267] In a bilateral scenario, the processor of the computer device housing the transmitting network interface card (NIC) does not need to exchange the aforementioned available memory addresses with the processor of the computer device housing the receiving NIC. Instead, the processor in the receiving computer device creates a mapping table and sends this mapping table to the receiving NIC. This mapping table records the association between the destination connection identifier and the available memory address in the memory of the receiving computer device. Furthermore, the data sent from the transmitting NIC to the receiving NIC carries the destination connection identifier. Therefore, after receiving the data, the receiving NIC can determine which available memory address in the receiving computer device's memory to store the data based on the destination connection identifier and the mapping table.
[0268] For example, in a two-sided scenario, during the process of a first computer device sending data to a second computer device, the first processor on the first computer device sends metadata to the first network interface card (NIC) through the SQ in the idle QP. This metadata indicates the storage address and length of the data to be sent by application process 1 in the memory of the first computer device. The first NIC can obtain the metadata through the SQ in the idle QP and retrieve the data to be sent from the memory of the first computer device based on the metadata. Alternatively, the metadata is carried by the SQE, which is sent by the first processor to the SQ, thus enabling the first processor to send the data to be sent to the first NIC using the SQE in the SQ. After obtaining the data to be sent, the first NIC encapsulates the data to be sent into a data packet and sends the data packet to the second NIC. After the data packet is sent successfully or after receiving the acknowledgment message corresponding to the data packet from the second NIC, the first NIC fills in a CQE in the CQ of the idle QP to notify the first processor that the data of application process 1 has been successfully sent. The second processor of the second computer device issues a mapping table, which records the data to be written to the memory of the second computer device after receiving data from the sending side (i.e., the first computer device). The second processor allocates a CQ to the second network interface card (NIC), which is shared by multiple application processes on the second computer device. After receiving a data packet sent by the first NIC, the second NIC decapsulates the data packet to obtain the corresponding data. The data is written to the memory (e.g., RAM) of the second computer device through the mapping table. A data reception completion message is then sent to the second processor via the CQ to notify the second processor that the data from application process 3 has been successfully received. The above data can be understood as the data transmitted between application process 1 on the first computer device and application process 3 on the second computer device.
[0269] Compared to traditional RDMA, this application achieves reliable connection sharing among multiple application processes by adding a QPG service layer. On one hand, the processors on both sides can assign connection identifiers (e.g., source or destination connection identifiers) to application processes. The header information of the data packets sent by the first network card includes the connection identifier, and the source and destination connection identifiers jointly identify a transmission channel. This means that the source and destination connection identifiers in the data packet can be used together as the connection carrier, eliminating the need for a specific QP. In other words, the data packet can carry transmission channel information without establishing a specific QP connection before sending the message; that is, the transmission channel information is unbound from a specific QP. This approach not only allows for flexible data packet transmission but also saves memory resources required to establish a specific QP. On the other hand, the QPG service layer can select an idle QP to transmit messages. Compared to existing long connections with fixed QPs, this method can also be called a message-based short connection. For example, after application process 1 finishes transmitting a message using QP1, QP1 is set to idle, allowing application process 2 to use QP1 to transmit messages, thus enabling multiple application processes to share a single QP.
[0270] The hardware system architecture and software architecture have been described above. The data communication method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0271] Please refer to Figure 4, which shows an embodiment of the data communication method provided in this application. This method can be executed by the first computer device and / or the second computer device shown in Figure 1 above.
[0272] The data communication method includes, but is not limited to, steps 401 to 414. Step 401: The first processor sends first information to the first network interface card (NIC). Step 402: The first NIC sends a first message to the second NIC based on the first QP (Queries Pointer). Step 403: The second NIC stores the first data in a second memory. Step 404: The first NIC establishes a first context. Step 405: The second NIC establishes a second context. Step 406: The second NIC sends a response message for the first message to the first NIC. Step 407: The first NIC sends second information to the first processor. Step 408: If a first preset condition is met, the first NIC deletes the first context. Step 409: The first processor or the first NIC changes the state of the first QP from non-idle to idle. Step 410: The second NIC sends fourth information to the second processor. Step 411: If a second preset condition is met, the second NIC deletes the second context. Step 412: The first NIC receives third information sent by the first processor. Step 413: The first network interface card (NIC) sends a second message to the second NIC based on the first QP. Step 414: The second NIC stores the second data in a second memory. Steps 404 to 414 are optional.
[0273] The steps described above are described in detail below.
[0274] Step 401: The first processor sends the first information to the first network card.
[0275] Different devices (such as the aforementioned computer devices or processors) can communicate via RDMA protocol through their respective network interface cards (NICs). For example, a first processor and a second processor can communicate via RDMA protocol through a first NIC and a second NIC. The first NIC and the second NIC can also be referred to as RDMA NICs or smart NICs.
[0276] In this configuration, the first network interface card (NIC) is located in the first computer device containing the first processor, and the second NIC is located in the second computer device containing the second processor. Alternatively, the first NIC is connected to the first processor, and the second NIC is connected to the second processor. The NICs can be connected to the processor in a plug-in manner, through chip integration, or via a bus, etc., etc., without specific limitations here.
[0277] Multiple application processes run on the first computer device, and these processes can communicate externally through the first network interface card (NIC). Specifically, the first processor acquires first data from the first application process among the multiple application processes and sends first information to the first NIC. Correspondingly, the first NIC receives the first information sent by the first processor. The aforementioned first information includes at least: the storage address of the first data and the length of the first data. The storage address may refer to a memory address in the first memory of the first computer device.
[0278] The aforementioned "first information" can also be understood as metadata sent by the first processor to the first network interface card (NIC). This process can also be understood as the transmission of first information between the first processor (where the NIC resides) and the first NIC. Furthermore, "first data" can refer to any data that a first application process on the first computer device wants to send. For example, the first data could be a message generated by a communication application and that needs to be sent.
[0279] Optionally, the first processor can also create a first QPG, which includes at least one QP. The first QPG is used for transmitting data from multiple application processes on the first computer device via RDMA. The first QPG being used for transmitting data from multiple application processes via RDMA can also be understood as multiple application processes on the first computer device sharing the first QPG. Furthermore, the process of creating at least one first QPG can be completed in one step or in multiple steps. For example, the computer device can create all the QPs in the first QPG at once. Alternatively, the computer device can first create some of the QPs in the first QPG, and then create new QPs for the first QPG based on data transmission needs or application process requests.
[0280] Optionally, after the first processor creates the first QPG, the application process on the first computer device can request a connection identifier from the first processor, which is used to represent the data transmission channel.
[0281] It should be noted that the first network interface card (NIC) may or may not be aware of the first QPG created by the first processor; this is not limited here. If the first NIC can be aware of the first QPG created by the first processor, the NIC may be aware of the first QPG in the following ways: the first processor maps the first QPG in the first NIC after creating it; or it may present the first QPG in the first NIC through mirroring; or the first processor may create the first QPG on the first NIC at the same time as creating the first QPG, etc., which is not limited here. The first QPG on the first processor and the first QPG on the first NIC can be understood as having a mirror relationship, a mapping relationship, or a channel relationship, etc.
[0282] For example, the first processor on the sending side can assign a first source connection identifier to the first application process. Correspondingly, the second processor on the receiving side can assign a first destination connection identifier to the third application process on the second computer device. Before transmitting the first data, the two processors can exchange the first destination connection identifier. The first processor can inform the first network interface card (NIC) of its assigned first source connection identifier and the first destination connection identifier assigned by the second processor. The first source connection identifier and the first destination connection identifier are used by the first NIC to transmit the first data to the second NIC. Alternatively, it can be understood that the first source connection identifier and the first destination connection identifier can jointly and uniquely represent a transmission channel. The method by which the second processor assigns the first destination connection identifier to the third application process is not limited. The second processor can directly assign the first destination connection identifier to the third application process, or it can create a second QPG service layer and assign the first destination connection identifier to the third application process through the second QPG service layer, etc. The specific method is not limited here.
[0283] It should be noted that during data transmission between the first and second processors, the first processor, acting as the sender, can indicate the data to be sent to the first network interface card (NIC) via the SQ (Signal Queued) in the QP (Queued Message), and can also receive data transmission completion information from the first NIC via the CQ (Confirmation Queued) in the QP. The second processor, acting as the receiver, may not be involved in SQ interaction and therefore does not need to create a QP. Furthermore, the second processor can also obtain data reception completion information through the CQ shared by multiple application processes. Of course, if the second processor is acting as the sender, it may also include the process of creating a QPG (Queued Message Group) and sending the data to be sent to the second NIC via the QP in the QPG.
[0284] For further descriptions of QPG, please refer to the description in the software architecture shown in Figure 2 above, which will not be repeated here.
[0285] It should be noted that the identifiers used in this application (e.g., source connection identifier, destination connection identifier, data identifier, etc.) can also be referred to as queue number, number, sequence number, order number, mark, symbol, sign, tag, marker, etc., without any specific limitation here. For example, the source connection identifier can also be called the source connection queue number (source CNQN), and the destination connection identifier can also be called the destination connection queue number (destination CNQN).
[0286] Optionally, the first information may also include a connection identifier for the first source end and a connection identifier for the first destination end. Of course, the first processor may also send the connection identifiers for the first source end and the first destination end to the first network card using information other than the first information; this is not limited here.
[0287] Furthermore, the first processor or the first network interface card (NIC) can also assign an identifier to the first data, which is used to uniquely identify the first data. It is understood that if the first processor assigns the identifier to the first data, the first information may also include the identifier of the first data. Similarly, the first processor can also send the identifier of the first data to the first NIC using information other than the first information; specific details are not limited here.
[0288] The identifier of the first data can be solely identified by the MSN, or it can be jointly identified by the MSN and the QPGN of the first QPG (or it can be understood as the MSN uniquely identifying a data in the first QPG). No specific limitation is made here. If the first data is jointly identified by the MSN and the QPGN of the first QPG, then when a QPG identifies the data, it does not need to consider whether it will conflict with the data identifiers in other QPGs.
[0289] Furthermore, in a unilateral scenario, the first processor and the second processor can also exchange the available memory address of the second memory, which is used to store the data (i.e., the first data) that the second network card will receive. After the first network card transmits a data packet carrying the available memory address of the second memory to the second network card, the second network card can determine which memory segment or storage area in the second memory to store the data in the data packet according to the available memory address of the second memory carried in the data packet.
[0290] The available memory address of the second memory can also be understood as the available memory address of the second memory in the second computer device. In a unilateral scenario, the first information may also include the available memory address in the second memory. Similarly, the first processor can also send the available memory address of the second memory to the first network card using information other than the first information; the specifics are not limited here.
[0291] It is understandable that if the first network interface card (NIC) senses the application process on the first computer device, the action of creating the first QPG can be performed by the first NIC, and the specifics are not limited here.
[0292] For example, the first processor places the first information in the SQ of the previously described QP, that is, the first information can be carried by the previously described SQE.
[0293] Step 402: The first network card sends a first message to the second network card based on the first QP.
[0294] After the first network interface card (NIC) acquires the first information, it can send a first message to the second NIC based on the first QP (Queries Pointer). Correspondingly, the second NIC receives the first message sent by the first NIC. For example, the process of the second NIC receiving the first message can also be understood as the process of storing the payload data of the first message in the second NIC's temporary memory.
[0295] The process of the first network interface card (NIC) sending a first packet to the second NIC based on the first QP (Queries Pointer) may include, but is not limited to, the following steps 1 to 3: Step 1: Obtain the first data based on the SQ (Single Queries) in the first QP. Step 2: Encapsulate the first data to obtain the first packet. Step 3: The first NIC sends the first packet to the second NIC. These steps are described below:
[0296] Step 1: The first processor puts the first information into the SQ of the QP, so that the first network card can obtain the first information through the SQ. Since the first information contains the storage address of the first data (e.g., the first storage address) and the length of the first data (e.g., the first length), after obtaining the first information, the first network card can obtain the data of the first length (i.e., the first data) from the first storage address.
[0297] Step 1 can be understood as follows: the user writes the first data into the memory of the first storage device, and the first processor notifies the first network card through SQ to copy the first data from the memory to the first network card.
[0298] It should be noted that the first network interface card (NIC) can acquire the first data in one or multiple transactions. For example, if the amount of the first data is small, the first NIC can directly copy all of the first data to its temporary storage. Alternatively, if the amount of the first data is large, the first NIC can copy the first data to its temporary storage in batches.
[0299] The concept of "small first data" can be understood as follows: the data volume of the first data is less than the preset threshold; or it can be understood as the first network interface card (NIC) being able to send the entire first data directly; or it can be understood as the first data requiring only a small payload; or it can be understood as the first data being transmitted through a small number of data packets. Conversely, the concept of "large first data" can be understood as the data volume of the first data being greater than or equal to the preset threshold; or it can be understood as the first NIC needing to slice the first data before sending; or it can be understood as the first data requiring a large payload; or it can be understood as the first data requiring transmission through a large number of data packets. The preset threshold can be set according to actual needs, or it can be related to the amount of data that the temporary storage space of the first NIC can store (e.g., the preset threshold is the upper limit of the amount of data that the temporary storage space can store), etc. Specific limitations are not specified here.
[0300] Step 2: Encapsulate the first data to obtain the first message.
[0301] After the first network interface card (NIC) acquires the first data, it can encapsulate the first data to obtain the first packet. Optionally, after the first NIC acquires the first data through the SQ in the first QP, it can encapsulate the first data to obtain the first packet. Specifically, the first NIC uses the first payload to carry the first data and adds the first header information to the first payload to obtain the first packet, and the state of the first QP is idle.
[0302] The first message includes a first payload and a first header. The first header includes: a connection identifier of the first source end, a connection identifier of the first destination end, and an identifier of the first data.
[0303] For ease of description, the three identifiers mentioned below include: the connection identifier of the first source end, the connection identifier of the first destination end, and the identifier of the first data.
[0304] The first payload can carry the entire first data or a portion of it. Furthermore, the connection identifier at the first source end and the connection identifier at the first destination end jointly identify a transmission channel. This means the connection carrier can be the source and destination connection identifiers in the data packet, eliminating the need for a specific QP. In other words, the data packet can carry transmission channel information without establishing a specific QP connection before sending the message; the transmission channel information is unbound from the specific QP. This approach not only allows for flexible data packet transmission but also saves memory resources required to establish a specific QP. Moreover, since a specific QP is not needed as a connection carrier, the second network interface card (NIC) can receive data packets without a QP, reducing the memory resources required to build a receiving-side QP for receiving packets. Alternatively, the reception of the first packet is independent of the QP.
[0305] Optionally, before acquiring the first data, the first network interface card (NIC) first determines the first QP (Queries Per Instance). There are several ways for the first NIC to determine the first QP:
[0306] In one possible implementation, the first processor determines an idle first QP within the first QPG and indicates, via first indication information, that the SQ in the first QP carries data to be transmitted. After receiving the first indication information, the first network interface card (NIC) can obtain the first data to be transmitted through the SQ in the first QP. The first indication information can be information from first information or information other than the first information; no specific limitation is made here. For example, the first information may also include the first indication information, and the first NIC, after receiving the first information, can determine the idle first QP through the first indication information.
[0307] In another possible implementation, the first network interface card (NIC) itself determines the first idle QP in the first QPG.
[0308] It should be noted that whether the first processor determines the first idle QP or the first network interface card (NIC) itself determines the first idle QP, the method of determining the first QP can be either to directly determine the first QP from at least one QP in the first QPG, or to create a new idle QP for the first QPG and use the newly created idle QP as the first QP, etc., the specific method is not limited here.
[0309] Furthermore, after determining the first QP, its state can be changed from idle to non-idle. By adjusting the state of the first QP, other application processes cannot use the non-idle first QP (or it can be understood as the first QP being a time-division multiplexed QP for multiple application processes). Of course, if the state of the first QP is subsequently adjusted to idle, other application processes can use this idle QP to transmit data with the network card. For example, after the first processor determines that the first QP is idle, it not only sends a first indication message to the first network card indicating that the first QP is idle, but also changes the state of the first QP from idle to non-idle. Alternatively, after the first network card itself determines that the first QP is idle, it changes the state of the first QP from idle to non-idle. Or, after the first processor determines that the first QP is idle and sends a first indication message to the first network card, the first network card changes the state of the first QP from idle to non-idle. Or, after the first network card itself determines that the first QP is idle, it reports to the first processor that it has obtained the first data to be transmitted from the SQ in the first QP, and the first processor changes the state of the first QP from idle to non-idle. That is, the state transition of the first QP can be executed by the first processor, the first network interface card (NIC), or both the first processor and the first NIC. It can also be understood that the entity executing the state transition of the first QP can be the same as or different from the entity determining the first QP.
[0310] In addition to the three identifiers mentioned above, the first header information may also include at least one of the following: the available memory address of the second memory, the identifier of the first QPG (i.e., QPGN), an offset, User Datagram Protocol (UDP) header information, Internet Protocol (IP) header information, and Ethernet (Eth) header information. The offset is used to indicate the position of the first transmitted message within multiple messages. Alternatively, it can be understood as indicating the position of the first payload within multiple first payloads, where the first data is carried by multiple first payloads. When the first data is transmitted through multiple datagrams, the second network interface card (NIC) can correctly concatenate multiple payloads using the offset in the first header information.
[0311] For example, the first network interface card (NIC) can obtain UDP header information, IP header information, and Eth header information through the Queue Pair Context (QPC) of the first QP. This QPC can be used to record at least one of the following: the number / data volume of SQEs sent by the SQ in the first QP, the number / data volume of SQEs to be sent by the SQ in the first QP, the number / data volume of RQEs received by the RQ in the first QP, the number / data volume of RQEs that the RQ can receive in the first QP, the number / data volume of CQEs received by the CQ in the first QP, and the number / data volume of CQEs that the CQ can receive in the first QP, etc.
[0312] For example, taking the first and second network interface cards (NICs) supporting the RoCE v2 protocol as an example, the RoCEv2 protocol stack includes: InfiniBand (IB) transport layer, UDP, IP, and Eth. The first header information in this application can refer solely to the header field of the IB transport layer, or it can refer to a header field that includes other header fields, etc., without specific limitations here. Other header fields include at least one of the following: Ethernet header field, IP header field, UDP header field, and IB transport layer header field, etc.
[0313] For example, assuming the first header information is 32 bits long (or can be understood as 32 bits), an example of the first message is shown in Figure 5.
[0314] The Base Transport Header (BTH) describes the transport layer of the message. Specifically, BTH may include: OpCode, S (or SE), M, Pad, Transport Header Version (TVer), Partition Key, first destination connection identifier (dest CNQN), and Packet Sequence Number (PSN). The Group Extended Transport Header (GETH) may include: first QPG identifier (QPGN), first source connection identifier (source CNQN), first data identifier (MSN), and offset.
[0315] OpCode is used to indicate the type of the packet or the protocol type of a higher layer in the IB payload. In this application, OpCode can also be used to indicate the QPG service type. This newly added QPG service type is used to indicate that the first header information includes the aforementioned three identifiers.
[0316] S is an abbreviation for Solicited Event, indicating that the responder should generate an event.
[0317] M is an abbreviation for MigReq, which is generally used for migration states.
[0318] Pad indicates how many extra bytes are padded into the IB payload.
[0319] TVer is used to indicate the version number of the first message.
[0320] The Partition Key is used to represent the logical memory partition associated with the first message.
[0321] `dest CNQN` is used to represent the connection queue of QPG on the receiving node (i.e., the second network interface card).
[0322] PSN is used to detect lost or duplicate data packets.
[0323] QPGN is used to uniquely represent a QPG.
[0324] The source CNQN is used to represent the connection queue of QPG on the sending node (i.e., the first network card).
[0325] MSN is used to uniquely represent the first data in QPG.
[0326] `offset` is used to represent the offset of a message within a message.
[0327] It is understandable that the first message may include other fields in addition to the first header information and the first payload; specific fields are not limited here. Other fields may include at least one of the following: fields for redundancy detection, fields for frame verification, header fields from other layers, etc.
[0328] Step 3: The first network card sends the first message to the second network card.
[0329] After the first network interface card (NIC) generates the first message, it sends the first message to the second NIC. Correspondingly, the second NIC receives the first message sent by the first NIC.
[0330] Step 403: The second network card stores the first data in the second memory.
[0331] After receiving the first message, the second network card can store the first data in the second memory of the second computer device where the second processor is located.
[0332] The second memory, also known as the RAM of a second computer device, is primarily used to store software programs and data. The second memory can exist independently and be connected to the second processor. Optionally, the second memory can be integrated with the second processor, for example, integrated within a single chip (i.e., the second memory can also be called an on-chip memory element). Furthermore, the second memory can be used by the second network interface card (NIC) to store or retrieve data.
[0333] For example, the second memory may include volatile memory, such as random access memory (RAM). The second memory may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).
[0334] Specifically, the second network card receives the first message, decapsulates the first message to obtain the first payload, and stores the data carried by the first payload in the second memory.
[0335] Optionally, the second network card can directly access the second memory without going through the second processor, thereby storing the received first data in the second memory.
[0336] There are several scenarios in which the second network interface card (NIC) stores the first data in the second memory. For example, the second NIC can first store the first data in its temporary storage, and then write the data from the temporary storage to the second memory after the temporary storage reaches a threshold or after a certain period of time since receiving the first packet. Alternatively, when receiving multiple packets, the second NIC can directly store the data from each received packet into the second memory. Yet another example is that, when receiving multiple packets, the second NIC can store the data from all the received packets into the second memory only after receiving all the packets.
[0337] Furthermore, the second network interface card (NIC) needs to know where the data carried by the first payload is stored in the second memory. The second NIC has several ways to obtain the storage location:
[0338] In one possible implementation, in a unilateral scenario, as described in step 402 above regarding the first header information, the first header information may include the available memory address of the second memory. After receiving the first packet, the second network interface card (NIC) can determine the available memory address of the second memory by parsing the first header information, and thus store the first data in the available memory address of the second memory.
[0339] In another possible implementation, in a bilateral scenario, the second processor can also establish a mapping table. This mapping table records the mapping relationship between the connection identifier of the first destination and the target storage address in the second memory (to distinguish it from the subsequent storage address related to the second data, this target storage address can be called the first target storage address, and the target storage space can be called the first target storage space). The first target storage address is used to indicate the first target storage space in the second memory. The connection identifier of the first destination can also be associated with a third application process among multiple application processes on the second computer device. The connection identifier of the first destination is used to store the first data. After establishing the mapping table, the second processor can send the mapping table to the second network card. The mapping table is used by the second network card to store the first data using the first target storage space. After receiving the mapping table sent by the second processor, the second network card stores the first data in the first target storage space based on the mapping table. In this method, the second network card can use the mapping table sent by the second processor to clearly specify which part of the second main memory to store the data. On the one hand, the second network card can store data without the participation of the second processor. On the other hand, the second processor can use the data stored by the second network card in the corresponding storage space through the sent mapping table.
[0340] The first target storage address can be either a physical address (PA) or a virtual address (VA), without any specific limitation here. Furthermore, each virtual address uniquely corresponds to one physical address.
[0341] Furthermore, the timing of the second network card obtaining the mapping table can be either before or after receiving the first packet; there is no limitation on the timing of the second network card obtaining the mapping table here.
[0342] For example, one example of the mapping table mentioned above is shown in Table 1:
[0343] Table 1
[0344] After receiving the first packet, the second network interface card (NIC) looks up the mapping table shown in Table 1 using the connection identifier of the first destination to determine the corresponding first target storage address. Then, it writes the first data to the first target storage address.
[0345] Understandably, if the first target storage address is a virtual storage address, the second network card can obtain the physical address by querying the Memory Translation Table (MTT) using the key (MR), virtual storage address, and length.
[0346] Of course, if the key (MR) of the mapping table indicates that writing is not allowed, a negative acknowledgment (NAK) is returned.
[0347] It should be noted that the aforementioned first target storage address can be determined based on the message receiving request generated by the third application process on the second computer device. This request can also be understood as a work request (WR). For example, when the third application process generates a message receiving request, it records a mapping table entry corresponding to the connection identifier of the first destination. Then, the application process on the second computer device inserts the entry into the mapping table through the driver. If the entry already exists, the count is incremented by 1.
[0348] It is understandable that the process of storing the first data into the address of the second memory (such as the aforementioned available memory address or target memory address) can also be understood as the process of the first network card "writing" to the memory of the second computer device. Of course, in practical applications, the address of the second memory (such as the aforementioned available memory address or target memory address) can also be used for "reading," that is, the second network card can copy the corresponding data from the second memory to the temporary storage of the second network card according to the above address, thereby transmitting the corresponding data to the first network card, so as to realize the process of the first network card reading the memory data in the remote second computer device.
[0349] Step 404: The first network interface card establishes the first context. This step is optional.
[0350] Optionally, the first network interface card (NIC) can also establish a first context for the first data. The first context is used to maintain the transmission status of the first data. For example, the first context for the first data includes at least one of the following: the amount of data already transmitted, and how much data remains to be transmitted. It is understood that the first context can also include the three identifiers mentioned above and / or an offset, where, in the case of the first data being transmitted through multiple packets, the offset is used to indicate the position of the transmitted first packet within the multiple packets.
[0351] Furthermore, the first network interface card (NIC) can establish a first context for the first data after acquiring the first data based on the first information. Alternatively, it can establish the first context for the first data after generating the first packet corresponding to the first data. It can also establish the first context for the first data after sending the first packet to the second NIC. The timing of establishing the first context is not limited here.
[0352] Optionally, if the first data corresponds to multiple packets, the first network interface card (NIC) needs to update the first context after sending each packet. This includes updating the amount of data already sent and how much data remains to be sent.
[0353] Understandably, the process of establishing the first context can be understood as a means to improve the reliability of data granularity (or message granularity).
[0354] Step 405: The second network interface card establishes a second context. This step is optional.
[0355] Optionally, after receiving the first packet, the second network interface card (NIC) can establish a second context for the corresponding first data. The second context is used to maintain the reception status of the first data; for example, the second context for the first data includes at least one of the following: the amount of first data that has been received, and how much data has not yet been received. It is understood that the second context may also include the three identifiers and / or offsets mentioned above.
[0356] Furthermore, the second network interface card (NIC) can establish a second context for the first packet after receiving it. Alternatively, it can establish a second context for the first data after decapsulating the first packet. It can also establish a second context for the first data after sending feedback information (such as an acknowledgment or negative acknowledgment) corresponding to the first packet to the first NIC. The timing of establishing the second context is not limited here.
[0357] Of course, the second context can be the first data, the first message, or the first payload in the first message; no specific limitation is made here.
[0358] For example, consider a first message consisting of message 1 and message 2. After receiving message 1, the second network interface card (NIC) records the three identifiers and offset in message 1. When message 2 is received, the corresponding context can be found using the three identifiers (e.g., the context identifier can be obtained through direct concatenation, hashing, or other algorithms). Then, the context can be used to check whether message 1 was received. For example, if message 1 is not received, the first NIC is notified to retransmit message 1.
[0359] Step 406: The second network card sends a response message for the first packet to the first network card. This step is optional.
[0360] Optionally, after correctly receiving the first message, the second network interface card (NIC) can send an acknowledgment message to the first NIC, indicating that the first message has been correctly received. It is understood that, in order for the first NIC to clearly identify which message corresponds to the acknowledgment message, the acknowledgment message may also carry the aforementioned three identifiers.
[0361] The feedback of acknowledgment messages facilitates other operations or subsequent operations for the first network interface card (NIC), and reduces timeout retransmissions. For example, after receiving an acknowledgment message for a certain packet, the first NIC can send subsequent packets to the second NIC, improving the transmission efficiency of the first data. As another example, the second NIC can delete the second context after sending the acknowledgment message for the last packet, reducing the power consumption incurred by the second NIC in maintaining the second context.
[0362] It should be noted that the content of the header information in the response message can be obtained from the first header information of the first message, or through other means, etc., and no specific limitation is made here.
[0363] Step 407: The first network card sends the second information to the first processor. This step is optional.
[0364] Optionally, the first network interface card (NIC) can also send second information to the first processor. The second information indicates that the first data transmission is complete. Similarly, this process can be understood as the transmission of second information between the first NIC and the first processor where the NIC resides. For example, this second information could refer to the data carried by the CQE in the completion queue (CQ) of the aforementioned first QP. Alternatively, it can be understood that the first NIC can report the completion of the first data transmission to the first computer device through the CQ in the first QP.
[0365] Furthermore, the first network interface card (NIC) can send the second information to the first processor after sending the first data packet. Alternatively, it can send the second information to the first processor after receiving the acknowledgment message corresponding to the first data packet. It can also send the second information to the first processor after deleting the first context. The timing of the first NIC sending the second information to the first processor is not limited here. Correspondingly, the second information can indicate that the first data transmission is complete, or it can indicate that the first data has been correctly received by the second NIC; the specific meaning is not limited here.
[0366] For example, the second information may include an indication message and the aforementioned three identifiers, wherein the three identifiers are used by the first processor to determine which data was transmitted through which transmission channel (or to be understood as the data transmission completion of which associated application process), and the indication message is used to indicate that the data transmission corresponding to the three identifiers has been completed.
[0367] Step 408: If the first preset condition is met, the first network card deletes the first context. This step is optional.
[0368] Optionally, after the first network interface card establishes the first context, the first context can be deleted if the first preset condition is met.
[0369] The first preset condition includes at least one of the following: all packets containing the first data have been sent; or, acknowledgment packets from the second network interface card (NIC) for all packets corresponding to the first data have been received. It is understood that, in the case where the first data is transmitted through multiple packets, the first preset condition could be receiving an acknowledgment packet for the last packet among the multiple packets.
[0370] As can be seen, the first network interface card (NIC) maintains the transmission status of the first data through the established first context, ensuring that the first data is transmitted correctly and that the second NIC can receive the first data completely. For example, in the event of a lost packet, the first context can be used to determine which packets need to be retransmitted, guaranteeing that the second NIC can reliably receive the first data. Furthermore, promptly deleting the data context after data transmission not only saves storage space but also reduces the power consumption of the first NIC.
[0371] Step 409: The first processor or the first network interface card (NIC) changes the state of the first QP from non-idle to idle. This step is optional.
[0372] Optionally, the first processor or the first network interface card (NIC) can change the state of the first QP from non-idle to idle. This facilitates the subsequent sending of other data through the idle first QP. It can be seen that the first QP is promptly released for other data transmission after sending data, thus achieving a short-lived data-based connection.
[0373] For example, let's take the first network interface card (NIC) performing this step. For instance, after sending the first packet corresponding to the first data, the first NIC can change the state of the first QP from non-idle to idle. Alternatively, after receiving feedback from the second NIC that the packet corresponding to the first data has been correctly received, the first NIC can change the state of the first QP from non-idle to idle. Another example is that after deleting the first context, the first NIC can change the state of the first QP from non-idle to idle. Yet another example is that after sending second information to the first processor via the CQ in the first QP, the first NIC can change the state of the first QP from non-idle to idle. Finally, after receiving a complete acknowledgment message from the second NIC, the first NIC can change the state of the first QP from non-idle to idle.
[0374] For example, taking the first processor performing this step as an example. The first processor first determines an idle first QP and sends first information to the first network card through the SQ in the first QP. Then, it changes the state of the first QP from idle to non-idle. After receiving the second information through the CQ in the first QP, the first processor changes the state of the first QP from non-idle to idle.
[0375] Step 410: The second network card sends the fourth message to the second processor. This step is optional.
[0376] Optionally, the second network interface card (NIC) can also send a fourth message to the second processor, which indicates that the first data reception is complete. Similarly, this process can also be understood as the transmission of the fourth message between the second NIC and the processor in the second computer device where the second NIC is located.
[0377] Furthermore, the second network interface card (NIC) can send the fourth information to the second processor after receiving the first data packet. Alternatively, it can send the acknowledgment message corresponding to the first data packet. It can also send the fourth information after deleting the second context. The timing of the second NIC sending the fourth information to the second processor is not limited here.
[0378] For example, the fourth information may include an indication information and the aforementioned three identifiers, wherein the three identifiers are used by the second processor to determine which data was transmitted through which transmission channel (or to associate with which application process has completed data reception), and the indication information is used to indicate that the data transmission corresponding to the three identifiers has been completed.
[0379] Furthermore, the second network interface card (NIC) can report to the second computer device via the fourth information that the first data corresponding to the connection identifier of the first destination has been transmitted and placed in the second memory. The second processor can determine, based on the connection identifier of the first destination, which application process has completed data transmission and which memory address it has been written to. The third application process on the second computer device can then retrieve the first data from the corresponding address. This enables the first application process on the first computer device to transmit the first data with the third application process on the second computer device.
[0380] Optionally, the second network interface card (NIC) reports the fourth information to the second computer device based on CQ. Alternatively, this can be understood as the fourth information being carried by CQE within CQ. CQ is used to transmit data reception completion information for multiple application processes on the second computer device.
[0381] For example, after the second processor creates a completion queue context (CQ), the driver of the second computer device synchronously creates a corresponding completion pair context (CQC) on the second network interface card (NIC). The second NIC carries data reception completion information through the CQE in the CQ pointed to by the CQC, so that the second processor can determine that data reception is complete through the corresponding CQE. This then notifies the application process to retrieve the data from the corresponding address.
[0382] The aforementioned CQC is used to determine at least one of the following: the number / data volume of CQEs already received by CQ, the remaining number / data volume of CQEs that CQ can receive, etc.
[0383] Optionally, an application process on the second computer device can request a destination connection identifier from the second processor. This connection identifier is used to represent the data transmission channel. For example, the second processor can allocate a first destination connection identifier to a third application process on the second computer device. Before receiving the first message, the second processor can exchange the first destination connection identifier with the first processor. Thus, the first processor can inform the first network interface card (NIC) of its allocated first source connection identifier and the first destination connection identifier allocated by the second processor. The first destination connection identifier is used by the first NIC to send the first message. Specifically, the first source connection identifier and the first destination connection identifier are used by the first NIC to transmit first data to the second NIC. Alternatively, it can be understood that the first source connection identifier and the first destination connection identifier can jointly and uniquely represent a transmission channel.
[0384] For example, Figure 6 illustrates an example process of a receiving-side data communication method. This example process includes, but is not limited to, steps 1 through 4.
[0385] Step 1: After receiving the first packet, the second network interface card (NIC) stores it in its temporary storage. Then, the second NIC pushes the address and length of the first packet from the temporary storage to the thread, allowing the thread to retrieve the first packet based on the address and length. The thread can then decapsulate the first packet to obtain the first header information and the first data carried by the first payload. The thread can be understood as the physical processing thread within the second NIC.
[0386] Step 2: The second network interface card (NIC) creates a second context. The second context will be described later and will not be elaborated on here.
[0387] Step 3: The first network card sends an acknowledgment response to the first network card, which indicates that the first network card has correctly received the first data.
[0388] Step 4: The second network card reports the completion of receiving the first data to the second computer device via CQ. CQ is used to transmit data reception completion information for multiple application processes on the second computer device.
[0389] In addition, step 1 has another processing form (not shown in Figure 6). For example, the second network card first stores the first packet in the temporary storage of the second network card, and performs partial decapsulation on the first packet to obtain the connection identifier of the first destination end in the first header information. Then, it loads the QPC corresponding to the connection identifier of the first destination end to the thread, and the thread performs full decapsulation to obtain the first data carried by the first payload.
[0390] Compared to the other processing method in step 1 above, the processing method shown in Figure 6 not only does not involve QP-related information (such as QPC), but also allows the first message to be decapsulated in one go through a thread.
[0391] For example, taking the second processor determining an idle QP as an example, Figure 7 illustrates another example process of the receiving-side data communication method. A third application process runs on the second computer device. The second processor can handle data requests from the third application process. For example, the memory stores the instructions of the third application process, and the second processor can call these instructions to run the third application. After the third application process generates WR1, it inserts the corresponding entry of WR1 into the mapping table through the driver. Taking the target storage address in the mapping table as VA as an example, the entry corresponding to WR1 includes: destination connection identifier-2, key(MR)-aa, VA-bb, length-ee, and count-1. The entry corresponding to WR2 includes: destination connection identifier-1, key(MR)-cc, VA-dd, length-ff, and count-1. After receiving the packet, the thread in the second network card determines the VA-bb associated with the destination connection identifier 2 from the mapping table through the destination connection identifier 2 carried in the packet, and then finds the PA corresponding to VA-bb through the MTT table, and then stores the M1 data carried by the payload into the corresponding PA in the physical memory space. Therefore, the third application process obtains the corresponding M1 data based on the PA. The thread shown in Figure 7 and the Direct Memory Access (DMA) engine can be collectively referred to as the aforementioned RDMA engine.
[0392] Step 411: If the second preset condition is met, the second network card deletes the second context. This step is optional.
[0393] Optionally, after the second network interface card establishes the second context, the second context can be deleted if the second preset condition is met.
[0394] The second preset condition includes at least one of the following: acknowledgment messages for all messages corresponding to the first data have been sent; or, fourth information has been sent to the second processor; the fourth information is used to indicate that the first data reception is complete.
[0395] It can be seen that by maintaining the reception status of the first data through the established second context, the second network interface card (NIC) can improve the correct reception of the first data and reduce the occurrence of over-reception or under-reception. Furthermore, promptly deleting the data context after data transmission is complete not only saves storage space but also reduces the power consumption of the second NIC.
[0396] Step 412: The first network card receives the third information sent by the first processor. This step is optional.
[0397] Optionally, the first processor can also send third information to the first network interface card (NIC). Accordingly, the first NIC receives the third information sent by the first processor, which includes: the storage address and length of the second data, wherein the first data and the second data are generated by different application processes. For example,
[0398] The third information, similar to the first information, can be understood as additional metadata sent by the first processor to the first network interface card (NIC). This process can also be understood as the transmission of the third information between the first NIC and the processor in the first computer device where the NIC resides. Furthermore, the second data can refer to any data (or any message) that another application process (e.g., a second application process) in the first computer device wants to send.
[0399] Optionally, after the first processor creates the first QPG, the second application process on the first computer device can request a source connection identifier from the QPG service layer, which is used to represent the data transmission channel.
[0400] For example, the QPG service layer in the first computer device (as the sending side) can allocate a second source connection identifier to the second application process. Correspondingly, the second processor in the second computer device (as the receiving side) can allocate a second destination connection identifier to the fourth application process on the second computer device. Before transmitting the second data, the two processors can exchange the second destination connection identifiers. Thus, the second processor can inform the first network interface card (NIC) of its allocated second source connection identifier and the second destination connection identifier. The second source connection identifier and the second destination connection identifier are used by the first NIC to transmit the second data to the second NIC. Alternatively, it can be understood that the second source connection identifier and the second destination connection identifier can jointly and uniquely represent another transmission channel.
[0401] Optionally, the third information may also include the connection identifiers of the second source end and the second destination end. Of course, the first processor may also send the connection identifiers of the second source end and the second destination end to the first network card using information other than the third information; this is not limited here.
[0402] Furthermore, the first processor or the first network interface card (NIC) can also assign an identifier to the second data, which uniquely identifies the second data. It is understood that if the first processor assigns the identifier to the second data, the third information may also include the identifier of the second data. Similarly, the first processor can also send the identifier of the second data to the first NIC using information other than the third information; specific details are not limited here.
[0403] The identifier for the second data can be determined solely by the MSN, or jointly by the MSN and the QPGN of the first QPG; the specific method is not limited here. If the second data identifier is jointly determined by the MSN and the QPGN of the first QPG, then a QPG does not need to consider whether it will conflict with the data identifiers in other QPGs when identifying the data.
[0404] Furthermore, in a unilateral scenario, the first processor and the second processor can also interact regarding the available memory address of the memory in the second computer device where the second processor resides. This available memory address is used to store the data (i.e., the second data) that the second network card will receive. Therefore, after the first network card on the first computer device transmits a data packet carrying the available memory address of the second memory to the second network card, the second network card can determine, based on the available memory address of the second memory carried in the data packet, which memory segment or storage area in the second memory should be stored next.
[0405] The available memory address of the second memory can also be understood as the available memory address of the memory in the second computer device. In a unilateral scenario, the third information may also include the available memory address in the second memory. Similarly, the first processor can also send the available memory address of the second memory to the first network card using information other than the third information; the specifics are not limited here.
[0406] Step 413: The first network interface card (NIC) sends a second message to the second NIC based on the first QP. This step is optional.
[0407] Optionally, the first network interface card (NIC) can send a second message to the second NIC based on the first QP. Correspondingly, the second NIC receives the second message sent by the first NIC. For example, the process of the second NIC receiving the second message can also be understood as the process of storing the payload data of the second message in the temporary memory of the second NIC.
[0408] Similar to the description of sending the first message in step 402 above, the process of the first network interface card (NIC) sending the second message to the second NIC based on the first QP can specifically include the following steps 1 to 3: Step 1: Obtain the second data based on the SQ in the first QP. Step 2: Encapsulate the second data to obtain the second message. Step 3: The first NIC sends the second message to the second NIC. The differences between the first message and the second message are described below:
[0409] Specifically, after the first network interface card (NIC) obtains the third information, since the third information contains the storage address and length of the second data, the first NIC can determine the data of the corresponding length from the storage address of the second data. For example, if the user writes the second data into the memory of the first storage device, the first processor notifies the first NIC via CQE to copy the second data from memory to the first NIC.
[0410] It should be noted that the first network interface card (NIC) can acquire the second data once or multiple times. For example, if the amount of the second data is small, the first NIC can directly copy all of the second data to its temporary storage. Alternatively, if the amount of the second data is large, the first NIC can copy the second data to its temporary storage in batches.
[0411] The term "smaller second data" can be interpreted as follows: the second data volume is less than a preset threshold; it can also be interpreted as the first network interface card (NIC) being able to send the entire second data directly; it can also be interpreted as the second data requiring only a small payload; or it can be transmitted using only a small number of data packets. Conversely, "larger second data" can be interpreted as the second data volume being greater than or equal to the preset threshold; it can also be interpreted as the first NIC needing to slice the second data before sending; it can also be interpreted as the second data requiring a large payload; or it can be interpreted as the second data requiring transmission using a large number of data packets. The preset threshold can be set according to actual needs, or it can be related to the amount of data that the first NIC's temporary storage space can store (e.g., the preset threshold is the upper limit of the amount of data that the temporary storage space can store), etc. Specific limitations are not specified here.
[0412] The second message includes a second payload and a second header. The second payload carries the second data, and the second header includes: a connection identifier for the second source end, a connection identifier for the second destination end, and an identifier for the second data. Alternatively, it can be understood that after the first network interface card (NIC) has transmitted the first data through the first QP (Quadrature Port), and the state of the first QP has changed from non-idle to idle, the first NIC can still transmit the second data through the idle first QP. That is, the first QP is not specific to any one application process, but can transmit data for multiple application processes.
[0413] It should be noted that the first network card can also transmit the second data through other idle QPs. The first QP is used as an example of an idle QP to illustrate that data generated by different application processes can be transmitted through the same idle QP.
[0414] For example, if a user writes second data into the memory of the first storage device, the first processor notifies the first network card via SQ to copy the second data from memory to the first network card. Correspondingly, the aforementioned third information can be carried in the SQE of SQ.
[0415] Similarly, the second payload can carry the entire second data or a portion of it. Furthermore, the connection identifier at the second source end and the connection identifier at the second destination end jointly identify another transmission channel. That is, the connection identifiers at the source and destination ends in the data packet can be used together as the connection carrier, without requiring a specific QP as the connection carrier. This can also be understood as the data packet carrying transmission channel information without needing to establish a specific QP connection before sending the message; that is, the transmission channel information is unbound from the specific QP. Through this method, not only can data packet transmission be flexibly implemented, but the memory resources required to establish a specific QP can also be saved.
[0416] Optionally, before sending the second message to the second network card based on the first QP, the first network card first determines the first QP. The method by which the first network card determines the first QP is similar to that in step 402 above; it can be either the first processor or the first network card itself determining the first QP. Specific details will not be elaborated here.
[0417] Optionally, after obtaining the third information, the first network interface card (NIC) can encapsulate the second data to obtain the second message. Specifically, the first NIC uses the second payload to carry the second data and adds second header information to the second payload to obtain the second message. After determining that a first available QP (Queries Per Second) is available, the second message is transmitted based on the first QP.
[0418] Similarly, like the first header information mentioned above, the second header information, in addition to including the connection identifier of the second source end, the connection identifier of the second destination end, and the identifier of the second data, may also include at least one of the following: the available memory address of the second memory (i.e., the second destination memory address), the identifier of the first QPG (i.e., QPGN), the offset, and other layer header information. The offset is used to indicate the position of the transmitted second packet within multiple packets. Alternatively, it can be understood that there are multiple second payloads, meaning the second data is carried by multiple second payloads, and the offset can also be understood as indicating the position of the second payload within the second data. When the second data is transmitted through multiple datagrams, the second network interface card (NIC) can correctly concatenate multiple payloads using the offset in the second header information.
[0419] For example, the structure of the second message is similar to that of the first message, with the main difference being the three different identifiers. For details, please refer to the description of the first message mentioned above, which will not be repeated here.
[0420] After the first network interface card (NIC) generates the second message, it sends the second message to the second NIC.
[0421] Step 414: The second network card stores the second data in the second memory of the second computer device where the second processor resides. This step is optional.
[0422] Optionally, after receiving the second message, the second network card can store the second data in the second memory.
[0423] Specifically, the second network card receives the second message, decapsulates the second message to obtain the second payload, and stores the data carried by the second payload in the second memory.
[0424] Optionally, the second network card can directly access the second memory without going through the second processor, thereby storing the received second data in the second memory.
[0425] Similarly, there are several scenarios in which the second network interface card (NIC) stores the first data in the second memory. For example, the second NIC can first store the first data in its temporary storage, and then write the data from the temporary storage to the second memory after the temporary storage reaches a threshold or after a certain period of time since receiving the first packet. Another example is that when receiving multiple packets, the second NIC can directly store the data from each received packet into the second memory. Yet another example is that, when receiving multiple packets, the second NIC can store the data from all the received packets into the second memory only after receiving all the packets.
[0426] Furthermore, the second network interface card (NIC) needs to know the location within the second memory where the data carrying the second payload will be stored. The second NIC can obtain the storage location in several ways:
[0427] In one possible implementation, in a unilateral scenario, as described in step 413 above regarding the second header information, the second header information may include the available memory address of the second memory. Therefore, after receiving the second packet, the second network interface card (NIC) can determine the available memory address of the second memory by parsing the second header information, and thus store the second data in the available memory address.
[0428] In another possible implementation, in a bilateral scenario, the second processor can also add entries to the mapping table. These added entries record the mapping relationship between the connection identifier of the second destination and the second target storage address, which indicates the second target storage space in the second memory. The connection identifier of the first destination can also be associated with a fourth application process among multiple application processes on the second computer device. The connection identifier of the second destination is used to store second data. After adding entries, the second processor can issue the added entries or updated mapping table to the second network interface card (NIC). The added entries or updated mapping table are used by the NIC to store the second data in the second target storage space. The second NIC stores the second data in the second target storage space based on the added entries or updated mapping table. In this method, the second NIC can specify which part of the second memory's storage space to store the data through the entries or mapping table issued by the second processor. On the one hand, the second NIC can store data without the second processor's involvement. On the other hand, the second processor can use the data stored by the second NIC in the corresponding storage space through the issued mapping table.
[0429] The second target storage address can be either PA or VA; no specific limitation is made here. Furthermore, each virtual address uniquely corresponds to one physical address.
[0430] Furthermore, the timing of the second network interface card (NIC) acquiring entries or mapping tables can be either before or after receiving the second packet; there is no limitation on the timing of the second NIC acquiring entries or mapping tables here.
[0431] For example, continuing with the example of the mapping table in Table 1 above, the mapping table after adding table entries can be shown in Table 2:
[0432] Table 2
[0433] After receiving the second packet, the second network interface card (NIC) looks up the corresponding second target storage address in the mapping table shown in Table 2 using the connection identifier of the second destination. Then, it writes the second data to the second target storage address.
[0434] It is understandable that if the second target storage address is a virtual storage address, the second network card can obtain the physical address by querying the MTT table using the key (MR), the virtual storage address, and the length.
[0435] Of course, if the key (MR) of the mapping table indicates that writing is not allowed, then NAK is returned.
[0436] It should be noted that the aforementioned second target storage address can be determined based on a message receive request generated by the fourth application process on the second computer device. This request can also be understood as a WR (Message Receiver). For example, when the fourth application process generates a message receive request, it records a mapping entry corresponding to the connection identifier of the second destination. Then, the second processor inserts the entry into the mapping table through the driver. If the entry already exists, the count is incremented by 1.
[0437] Furthermore, after receiving the second message, the second network interface card (NIC) sends a fifth message indicating that the second data reception is complete to the second processor. Similarly, the second NIC can send a data reception completion message to the second processor via a CQ shared by multiple application processes.
[0438] Optionally, the fifth information may also include a connection identifier of the second destination. The second processor can determine that the data reception of the fourth application process has been completed based on the connection identifier of the second destination, so that the fourth application process can obtain the second data from the second memory.
[0439] It is understood that, similar to steps 404 to 413 mentioned above, at least one of the following steps may also be included: the first network card and / or the second network card establish a context for the second data, the first network card and / or the second network card delete the context of the second data, the first network card and the second network card interact with the response message corresponding to the second message, the first network card and the first processor interact with the idle QP in the first QPG to complete the CQE of the second data transmission, the second network card and the second processor interact with the CQ to complete the CQE of the second data transmission, etc., and the specifics are not limited here.
[0440] It should be noted that the steps shown in Figure 4 have no timing restrictions. For example, step 404 can be before or after step 405; step 406 can be before or after step 410; step 409 can be before or after step 408; and step 411 can be before or after step 410.
[0441] This application provides a data communication method with various possibilities, meaning that steps 404 to 414 are optional. Specifically, it can be selected whether to include at least one of steps 404 to 414 based on actual needs. For example, the data communication method provided by this application includes steps 401 to 403. Another example is that the data communication method provided by this application includes steps 401 to 405. Yet another example is that the data communication method provided by this application includes steps 401 to 407. Yet another example is that the data communication method provided by this application includes steps 401 to 407 and step 410. Yet another example is that the data communication method provided by this application includes steps 401 to 411. Yet another example is that the data communication method provided by this application includes steps 401 to 414, and so on.
[0442] In this application, the first network interface card (NIC) can send data packets through an idle first QP (Quick Pointer). The data packet header includes the source connection identifier, the destination connection identifier, and the message sequence number of the payload data. The source and destination connection identifiers jointly identify a transmission channel, meaning the connection carrier can be the source and destination connection identifiers in the data packet, eliminating the need for a specific QP. This can also be understood as the data packet carrying transmission channel information, without needing to establish a connection to a specific QP before sending the message; that is, the transmission channel information is unbound from a specific QP. This method not only allows for flexible data packet transmission but also saves memory resources required to establish a specific QP. Furthermore, since the transmission channel information is unbound from a specific QP, this method can also be called a message-based short connection compared to existing long connections with fixed QPs. For example, after application process 1 transmits a message using QP1, QP1 is set to idle, allowing application process 2 to also use QP1 to transmit messages, thus enabling multiple application processes to share a single QP.
[0443] Furthermore, all application processes share a single QPG for reliable connections. Each message can be sent and received using any QP, and data addressing can be completed by consuming any RQE. In other words, the number of QPs is independent of the connection scale. All application processes on the same node share a single QP resource pool and communicate with QP pools on other nodes through this pool. Simultaneously, the concurrency of multiple QPs improves data transmission performance.
[0444] Furthermore, by maintaining the transmission state of the first data through the established context (e.g., the first network interface card (NIC) maintains the sending state of the first data through the first context, and the second NIC maintains the receiving state of the first data through the second context), the first data is ensured to be sent correctly, and the second NIC can receive the first data completely. For example, in the event of a lost packet, the first context can be used to determine which packets need to be retransmitted, guaranteeing that the second NIC reliably receives the first data. Additionally, promptly deleting the data context after data transmission not only saves storage space but also reduces NIC power consumption. Moreover, based on this reliable data transmission mechanism, the connection's lifecycle is the same as the data's lifecycle. This enables short-lived data connections, mitigating connection switching overhead.
[0445] To further clarify the differences between the above data communication methods in unilateral and bilateral scenarios, the two scenarios will be described separately below.
[0446] The transmitting node includes a transmitting processor and a transmitting network interface card (NIC). The receiving node includes a receiving processor containing memory and a receiving NIC.
[0447] For example, the specific process of data transmission in a unilateral scenario may include the following steps one through ten:
[0448] Step 1: The sending processor creates a QPG, identified by a unique QPGN. The receiving processor creates a CQ shared by multiple application processes.
[0449] Step 2: The sending process requests one or more source connection queues from the sending processor, each uniquely identified by a CNQN.
[0450] Step 3: The receiving process requests one or more destination connection queues from the receiving processor, each with a unique CNQN identifier.
[0451] Step 4: The sending processor and the receiving processor negotiate {the receiving QPG and CNQN}.
[0452] Step 5: The sending processor and the receiving processor negotiate the data address to be read / written in the memory of the computer device where the receiving processor is located.
[0453] Step 6: The sending processor application issues a read / write request. The message scheduler in the sending QPG selects an idle QP according to the specified scheduling policy and assigns the message to that QP for transmission.
[0454] Step 7: The selected QP at the sending end assembles the message and creates the message context on the sending end's network card, and then sends the message out.
[0455] Step 8: After receiving a packet, the receiving network interface card (NIC) creates a message context (only for the first packet). For read operations, the receiving NIC is responsible for reading data from the specified address, assembling an ACK packet containing the data, and replying to the sending NIC. For write operations, the receiving NIC is responsible for writing data to the specified address, assembling an ACK packet, and replying to the sending NIC.
[0456] Step 9: For read operations, when the sending end QP receives the message packet, it sends the data. If it is the tail packet, it deletes the message context and sends CQE to the sending end computer device. For write operations, after the sending end network card receives the tail packet ACK message, it deletes the message context and sends CQE to the sending end computer device.
[0457] Step 10: The sending end and the receiving end exchange information to inform each other that data transmission has ended.
[0458] The timing of the above steps is not limited here; for example, step two can be before or after step three.
[0459] For example, the specific process of data transmission in a bilateral scenario may include the following steps one through thirteen:
[0460] Step 1: The sending node creates a QPG, identified by a unique QPGN. The receiving node creates a CQ shared by multiple application processes.
[0461] Step 2: The sending process requests one or more source connection queues from the sending processor, each uniquely identified by a CNQN.
[0462] Step 3: The receiving process requests one or more destination connection queues from the receiving processor, each with a unique CNQN identifier.
[0463] Step 4: The sending processor and the receiving processor negotiate {the receiving QPG and CNQN}.
[0464] Step 5: The receiving process sends a mapping table (MR) to the receiving network card. The mapping table records the mapping relationship between the connection identifier of the destination and the memory address of the computer device where the receiving processor is located.
[0465] Step 6: The sending process fills the connection queue with the message to be sent.
[0466] Step 7: The message scheduler in the sending QPG selects an idle QP according to the specified scheduling policy and assigns the message to that QP for transmission.
[0467] Step 8: The selected QP at the sending end assembles the message on the network card, creates a message context, and then sends the message out (it may be split into multiple messages depending on the message size).
[0468] Step 9: After the receiving network card receives the packet, it creates a message context (only for the first packet), queries the MR mapping table to find the corresponding MR, and writes it to the specified memory.
[0469] Step 10: After receiving the tail packet of the message, the receiving network card deletes the message context, replies with a tail packet ACK to the sending network card, and sends a CQE to the receiving computer device.
[0470] Step 11: After receiving the CQE, the receiving processor places the message into the designated queue according to the destination connection queue number specified in the message.
[0471] Step 12: The receiving application process reads the message from the corresponding connection queue.
[0472] Step 13: After receiving the tail packet ACK message, the sending network card deletes the message context and sends it to CQE.
[0473] The timing of the above steps is not limited here; for example, step two can be before or after step three.
[0474] The data communication method in the embodiments of this application has been described above. The network card in the computer device provided in the embodiments of this application is described below. Please refer to Figure 8. One embodiment of the network card in the embodiments of this application includes: a transceiver unit 801 and a processing unit 802.
[0475] In one possible implementation, the network interface card (NIC) shown in FIG8 is the first NIC in the embodiments shown in FIG1 to FIG7, and the first NIC is located in a first computer device containing a first processor. In this case, the functions of each unit are as follows:
[0476] The transceiver unit 801 is used to receive first information sent by the first processor. The first information includes the storage address of the first data and the length of the first data.
[0477] Processing unit 802 is configured to send a first message to a second network interface card (NIC) in a second computer device based on a first queue (QP). The first message includes a first payload and a first header. The first header includes a connection identifier of a first source end, a connection identifier of a first destination end, and an identifier for first data. The first payload carries the first data. The identifier for the first data uniquely identifies the first data. Furthermore, the first QP is in an idle state.
[0478] In one possible implementation, processing unit 802 is further configured to determine an idle first QP within the first QPG. Processing unit 802 is also configured to change the state of the first QP from idle to non-idle.
[0479] In one possible implementation, the first information further includes first indication information, which is used to indicate that the first data is sent through the first QP.
[0480] In one possible implementation, the processing unit 802 is further configured to establish a first context for the first data, the first context being used to maintain the transmission status of the first data. The processing unit 802 is also configured to delete the first context if a first preset condition is met; the first preset condition includes at least one of the following: a message that has finished transmitting the first data; or, receiving an acknowledgment message from the second network interface card for all messages corresponding to the first data.
[0481] In one possible implementation, the transceiver unit 801 is further configured to receive a response message corresponding to the first message from the second network card, the response message being used to indicate that the second network card has correctly received the first message.
[0482] In one possible implementation, the first QP includes: a transmit queue SQ and a complete queue CQ; the SQ carries first information; a processing unit 802 is specifically used to obtain the first data based on the SQ; a processing unit 802 is specifically used to encapsulate the first data with a first network interface card (NIC) to obtain the first packet; and a processing unit 802 is specifically used to send the first packet to a second NIC. The transceiver unit 801 is further used to send second information to a first processor based on the CQ in the first QP, the second information indicating that the first data transmission is complete.
[0483] In one possible implementation, the second information mentioned above includes the connection identifier of the first source end.
[0484] In one possible implementation, the processing unit 802 is also used to change the state of the first QP from non-idle to idle.
[0485] In one possible implementation, the transceiver unit 801 is further configured to receive third information sent by the first processor. The third information includes: the storage address of the second data and the length of the second data. The first data and the second data are generated by different application processes. The processing unit 802 is further configured to send a second message based on the first QP. The second message includes a second payload and second header information. The second payload is used to carry the second data, and the second header information includes: a connection identifier of the second source end, a connection identifier of the second destination end, and an identifier of the second data. The SQ is also used to carry the first information.
[0486] In one possible implementation, the aforementioned first data corresponds to multiple messages, and the first header information also includes an offset, which is used to indicate the position of the transmitted first message among the multiple messages. Alternatively, it can be understood that there are multiple first payloads, that is, the first data is carried by multiple first payloads, and the offset can also be understood as indicating the position of the first payload within the first data.
[0487] In one possible implementation, the aforementioned first information may also include an identifier for the first data. Similarly, the first data may be identified solely by MSN, or it may be jointly identified by MSN and QPGN. Furthermore, the first information may also include a connection identifier for the first source end and a connection identifier for the first destination end.
[0488] In one possible implementation, the first QP mentioned above belongs to a queue pair group first QPG, which is created by a first processor and is used to transfer data of multiple application processes on a first computer device based on RDMA.
[0489] In one possible implementation, the aforementioned first QPG is a QPG shared by multiple application processes, including a first application process and a second application process. The first data is generated by the first application process, and the second data is generated by the second application process.
[0490] In one possible implementation, the processing unit 802 is further configured to determine the identifier of the first data, which is used by the first network interface card to send the first message.
[0491] In one possible implementation, the first information and the first header information also include the available memory address of the memory in the second computer device where the second network card is located, and the available memory address is used to store the first payload.
[0492] In one possible implementation, the network interface card (NIC) also includes a storage unit for storing the initial data.
[0493] In this embodiment, the operations performed by each unit in the network card are similar to those described in the first network card in the embodiments shown in Figures 1 to 7 above, and will not be repeated here.
[0494] In this embodiment, the transceiver unit 801 can send data packets through an idle first QP. The data packet header includes the source connection identifier, the destination connection identifier, and the message sequence number of the payload data. Since the source and destination connection identifiers jointly identify a transmission channel, the connection carrier can be defined by both the source and destination identifiers in the data packet, eliminating the need for a specific QP. This can also be understood as the data packet carrying transmission channel information, without needing to establish a connection to a specific QP before sending the message; that is, the transmission channel information is unbound from the specific QP. This approach not only allows for flexible data packet transmission but also saves memory resources required to establish a specific QP. Furthermore, because the transmission channel information is unbound from the specific QP, this method can be called a message-based short connection compared to existing long connections with fixed QPs. For example, after application process 1 transmits a message using QP1, QP1 is set to idle, allowing application process 2 to also use QP1 to transmit messages, thus enabling multiple application processes to share a single QP.
[0495] In another possible implementation, the network interface card shown in Figure 8 is the second network interface card in the embodiments shown in Figures 1 to 7 above. The second network interface card is located in a second computer device containing storage units, and the functions of each unit are as follows:
[0496] Transceiver unit 801 is used to receive a first message from a first network interface card (NIC). The first message includes a first payload and a first header. The first header includes a connection identifier of a first source end, a connection identifier of a first destination end, and an identifier of first data. The first payload carries the first data. Processing unit 802 is used to store the first data in a storage unit. The reception of the first message does not use a queue pair (QP).
[0497] In one possible implementation, processing unit 802 is further configured to establish a second context for the first data, which is used to maintain the reception status of the first data. For example, the second context for the first data includes at least one of the following: the amount of first data that has been received, and how much data has not yet been received. Of course, if a second preset condition is met, processing unit 802 can also be configured to delete the second context. The second preset condition includes at least one of the following: acknowledgment messages for all messages corresponding to the first data have been sent; or, fourth information has been sent to the second processor; the fourth information is used to indicate that the reception of the first data is complete.
[0498] In one possible implementation, the transceiver unit 801 is further configured to send an acknowledgment message for the first message to the first network interface card (NIC), the acknowledgment message indicating that the first message has been correctly received. It is understood that, in order for the first NIC to clearly identify which message corresponds to the acknowledgment message, the acknowledgment message may also carry the three identifiers from the aforementioned first header information corresponding to the first message.
[0499] In one possible implementation, transceiver unit 801 is further configured to send fourth information to the second processor, the fourth information indicating that the first data reception is complete. Similarly, this process can also be understood as the transmission of fourth information between the second network interface card (NIC) and the second processor in the second computer device where the NIC is located. For example, transceiver unit 801 is specifically configured to send fourth information to the second processor based on CQ. CQ is used to transmit data reception completion information for multiple application processes on the second computer device. That is, the fourth information can refer to CQE.
[0500] In one possible implementation, the fourth piece of information mentioned above includes a connection identifier for the first destination. The connection identifier for the first destination is used to clarify that the first data is associated with a third application process among multiple application processes.
[0501] In one possible implementation, the transceiver unit 801 is further configured to receive a second message, the second message including a second payload and second header information. The second header information includes: a connection identifier of a second source end, a connection identifier of a second destination end, and an identifier of second data. The second payload carries the second data, which corresponds to a different application process than the first data. The reception of the second message may not use QP.
[0502] In one possible implementation, the transceiver unit 801 is further configured to send a fifth message to the second processor via CQ, the fifth message being used to indicate that the reception of the second data is complete.
[0503] In one possible implementation, the fifth piece of information may also include the connection identifier of the second destination.
[0504] In one possible implementation, the aforementioned first data corresponds to multiple messages, and the first header information also includes an offset, which is used to indicate the position of the first message among the multiple messages. Alternatively, it can be understood that there are multiple first payloads, that is, the first data is carried by multiple first payloads, and the offset can also be understood as indicating the position of the first payload within the first data.
[0505] In one possible implementation, the transceiver unit 801 is further configured to receive a mapping table sent by the second processor. The mapping table records the mapping relationship between the connection identifier of the first destination and the target storage address. The target storage address is used to indicate the target storage space of the memory in the second computer device. The processing unit 802 is specifically configured to store the first data in the storage unit, specifically including: the second network card storing the first data in the target storage space based on the mapping table.
[0506] In this embodiment, the operations performed by each unit in the network card are similar to those described in the second network card in the embodiments shown in Figures 1 to 7 above, and will not be repeated here.
[0507] In this embodiment, the transceiver unit 801 can carry the source connection identifier, the destination connection identifier, and the message sequence number of the payload data in the header information of the received data packet. Furthermore, the source connection identifier and the destination connection identifier jointly identify a transmission channel, meaning that the source and destination connection identifiers in the data packet can be used together as the connection carrier, without requiring a specific QP as the connection carrier. This can also be understood as the data packet carrying transmission channel information without needing to establish a specific QP connection before sending the message. For example, the second network interface card can receive data and complete the information without using a specific QP, thus unbinding the transmission channel information from the specific QP. Through this method, not only can data packet transmission be flexibly implemented, but the memory resources required to establish a specific QP can also be saved. In addition, since the data of the transmission channel is unbound from the specific QP, compared to the existing long connections with fixed QPs, this method can also be called a data-based short connection.
[0508] The computer device provided in the embodiments of this application is described below. Referring to FIG9, one embodiment of the computer device in this application includes: a transceiver unit 901 and a processing unit 902. The computer device shown in FIG9 is the first computer device in the embodiments shown in FIG1 to FIG7 above. This first computer device can be understood as a device acting as a data sending end. In this case, the functions of each unit are as follows:
[0509] Processing unit 902 is used to send first information to transceiver unit 901, the first information including: the storage address of first data and the length of first data;
[0510] The transceiver unit 901 is also used to send a first message to the second network interface card based on the first queue to the QP. The first message includes a first payload and first header information. The first header information includes: a connection identifier of the first source end, a connection identifier of the first destination end, and an identifier of the first data. The first payload is used to carry the first data. In addition, the state of the first QP is idle.
[0511] In one possible implementation, the processing unit 902 is further configured to determine an idle first QP in the first queue pair group (QPG) and change the state of the first QP from idle to non-idle. The first information also includes first indication information, which is used to indicate that first data is to be sent through the first QP.
[0512] In one possible implementation, the processing unit 902 is further configured to change the state of the first QP from idle to non-idle after determining the first QP.
[0513] In one possible implementation, the first QP includes: a SQ and a CQ; the SQ carries first information; a transceiver unit 901 is specifically used to acquire the first data based on the SQ; the transceiver unit 901 is specifically used to encapsulate the first data to obtain the first message; the transceiver unit 901 is specifically used to send the first message to the second network interface card. Furthermore, the transceiver unit 901 is also used to send second information to the processing unit 902 based on the CQ, the second information indicating that the first data transmission is complete. Correspondingly, the processing unit 902 receives the second information sent by the transceiver unit 901. For example, this second information could refer to a CQE.
[0514] In one possible implementation, the second information mentioned above includes a connection identifier of the first source end, which is associated with the first application process.
[0515] In one possible implementation, processing unit 902 or transceiver unit 901 is further configured to change the state of the first QP from non-idle to idle; processing unit 902 is further configured to send third information to transceiver unit 901, the third information including: the storage address of the second data, the length of the second data, and the first data and the second data being generated by different application processes; transceiver unit 901 is further configured to send a second message to the second network interface card based on the first QP, the second message including a second payload and second header information, the second header information including: the connection identifier of the second source end, the connection identifier of the second destination end, and the identifier of the second data, the second payload being used to carry the second data. The SQ in the first QP is used to carry the third information.
[0516] In one possible implementation, the first QP belongs to a first queue pair group QPG, which is used to transfer data between multiple application processes on a computer device based on RDMA. The processing unit 902 is also used to create the first QPG.
[0517] In one possible implementation, the transceiver unit 901 is further configured to establish a first context for the first data, which is used to maintain the transmission status of the first data. For example, the first context for the first data includes at least one of the following: the amount of data already transmitted, and how much data remains to be transmitted. It is understood that the first context may also include an offset, which, when the first data is transmitted through multiple packets, indicates the position of the transmitted first packet within the multiple packets. Of course, if a first preset condition is met, the transceiver unit 901 can also be configured to delete the first context; the first preset condition includes at least one of the following: all packets containing the first data have been transmitted; or, acknowledgment messages have been received from all packets corresponding to the first data from the second network interface card. It is understood that, when the first data is transmitted through multiple packets, the first preset condition may be receiving an acknowledgment message from the last packet among the multiple packets.
[0518] In one possible implementation, the transceiver unit 901 is further configured to receive an acknowledgment message corresponding to the first message from the second network interface card (NIC), the acknowledgment message indicating that the second NIC has correctly received the first message. It is understood that, in order for the first NIC to clearly identify which message corresponds to the acknowledgment message, the acknowledgment message may also carry the three identifiers from the aforementioned first header information corresponding to the first message.
[0519] In one possible implementation, the first information mentioned above further includes: a connection identifier of the first destination, which is used by the first network interface card to send the first data.
[0520] In one possible implementation, the first information further includes: a connection identifier of the first source end, which is used by the first network interface card to send the first data; the processing unit 902 can also allocate a connection identifier of the first source end to the transmission channel, which, together with the connection identifier of the first destination end, identifies the transmission channel, and the transmission channel is used to transmit the first data.
[0521] In one possible implementation, the processing unit 902 may further determine the available memory address of the memory in the second computer device; the first information also includes the available memory address, the storage space indicated by the available memory address being used to store the first data. For example, the processing unit 902 interacts with the second processor to determine the available memory address of the memory in the second computer device. The processing unit 902 can indicate the available memory address of the memory in the second computer device to the first network interface card (NIC) through the first information, so that the first NIC can carry the available memory address of the second computer device in the first header information, so that after receiving the first message, the second NIC can store the first data carried by the first payload according to the address carried in the first header information.
[0522] In one possible implementation, the aforementioned first information may also include an identifier for the first data. Similarly, the first data may be identified solely by MSN, or it may be jointly identified by MSN and QPGN. Furthermore, the first information may also include a connection identifier for the first source end and a connection identifier for the first destination end.
[0523] In this embodiment, the operations performed by each unit in the computer device are similar to those described in the embodiments shown in Figures 1 to 7 above, and will not be repeated here.
[0524] In this scheme, processing unit 902 can send a data packet corresponding to the first data through an idle first QP. The data packet header includes the source connection identifier, the destination connection identifier, and the message sequence number of the payload data. Since the source and destination connection identifiers jointly identify a transmission channel, the connection carrier can be the source and destination connection identifiers in the data packet, eliminating the need for a specific QP. This can also be understood as the data packet carrying transmission channel information, without needing to establish a connection to a specific QP before sending the message; that is, the transmission channel information is unbound from the specific QP. This method not only allows for flexible data packet transmission but also saves memory resources required to establish a specific QP. Furthermore, since the transmission channel information is unbound from the specific QP, this method can also be called a message-based short connection compared to existing long connections with fixed QPs. For example, after application process 1 transmits a message using QP1, QP1 is set to idle, allowing application process 2 to also use QP1 to transmit messages, thus enabling multiple application processes to share a single QP.
[0525] Please refer to Figure 10. Another embodiment of the computer device in this application includes a transceiver unit 1001 and a storage unit 1002. The computer device shown in Figure 10 is the second computer device in the embodiments shown in Figures 1 to 7 above. This second computer device can be understood as a device acting as a data receiving end. In this case, the functions of each unit are as follows:
[0526] The transceiver unit 1001 is used to receive a first message from the first network interface card. The first message includes a first payload and first header information. The first header information includes: a connection identifier of the first source end, a connection identifier of the first destination end, and an identifier of the first data. The first payload is used to carry the first data. The reception of the first message does not use a queue pair (QP).
[0527] Storage unit 1002 is used to store the first data received by transceiver unit 1001.
[0528] In one possible implementation, the computer device further includes a processing unit, and a transceiver unit 1001 is also used to send fourth information to the processing unit, the fourth information being used to indicate that the first data transmission is complete.
[0529] In one possible implementation, the transceiver unit 1001 is specifically used to send a fourth message to the processing unit via a completion queue (CQ). The CQ is used to transmit data reception completion information for multiple application processes on the computer device. For example, this fourth message could refer to a CQE (Content Query Entrance).
[0530] The transceiver unit 1001 is also used to receive a second message from the first network interface card. The second message includes a second payload and second information. The second header information includes: a connection identifier of the second source end, a connection identifier of the second destination end, and an identifier of the second data. The second payload is used to carry the second data. The first data and the second data correspond to different application processes on the computer device (for example, the first data corresponds to the third application process, and the second data corresponds to the fourth application process).
[0531] The storage unit 1002 is also used to store the second data received by the transceiver unit 1001.
[0532] In one possible implementation, the transceiver unit 1001 is also used to report fifth information to the processing unit via CQ, the fifth information being used to indicate the completion of receiving the second data.
[0533] In one possible implementation, the transceiver unit 1001 is further configured to establish a second context for the first data. This second context is used to maintain the reception status of the first data. For example, the second context for the first data includes at least one of the following: the amount of first data already received, and the amount of data yet to be received. It is understood that the second context may also include an offset, which, when the first data is transmitted through multiple messages, indicates the position of the transmitted first message within the multiple messages. Of course, after the transceiver unit 1001 establishes the second context for the first data, if a second preset condition is met, the transceiver unit 1001 may also delete the second context. The second preset condition includes at least one of the following: acknowledgment messages for all messages corresponding to the first data have been sent; or, fourth information has been sent to the processing unit; the fourth information indicates that the reception of the first data is complete.
[0534] In one possible implementation, the transceiver unit 1001 is further configured to send an acknowledgment message for the first message to the first network interface card (NIC), the acknowledgment message indicating that the first message has been correctly received. It is understood that, in order for the first NIC to clearly identify which message corresponds to the acknowledgment message, the acknowledgment message may also carry the three identifiers from the aforementioned first header information corresponding to the first message.
[0535] In one possible implementation, the fourth piece of information mentioned above includes a connection identifier for the first destination. The connection identifier for the first destination is used to clarify that the first data is associated with a third application process among multiple application processes.
[0536] In one possible implementation, the fifth piece of information mentioned above includes a connection identifier for the second destination. The connection identifier for the second destination is used to clarify that the second data is associated with a fourth application process among multiple application processes.
[0537] In one possible implementation, the aforementioned first data corresponds to multiple messages, and the first header information also includes an offset, which is used to indicate the position of the first message among the multiple messages. Alternatively, it can be understood that there are multiple first payloads, that is, the first data is carried by multiple first payloads, and the offset can also be understood as indicating the position of the first payload within the first data.
[0538] In one possible implementation, the processing unit may also assign a first destination connection identifier to the transmission channel. The first destination connection identifier and the first source connection identifier together identify the transmission channel, which is a channel for transmitting the first data.
[0539] In this embodiment, the operations performed by each unit in the computer device are similar to those described in the embodiments shown in Figures 1 to 7 above, and will not be repeated here.
[0540] In this scheme, the header information of the data packets received by the transceiver unit 1001 carries the connection identifier of the source end, the connection identifier of the destination end, and the message sequence number of the payload data. Furthermore, the connection identifier of the source end and the connection identifier of the destination end jointly identify a transmission channel. This means that the connection identifiers of the source end and the destination end in the data packet can be used together as the connection carrier, without requiring a specific QP as the connection carrier. It can also be understood that the data packet can carry the transmission channel information without establishing a specific QP connection before sending the message; that is, the transmission channel information is unbound from a specific QP. For example, the transceiver unit 1001 can receive data packets without using a QP. Through this method, not only can data packet transmission be flexibly implemented, but the memory resources required to establish a specific QP can also be saved. In addition, since the reception of data packets is independent of the QP, compared to the existing long connections with fixed QPs, this method can also be called a data-based short connection.
[0541] Figure 11 is a schematic diagram of a computer device provided in an embodiment of this application. As shown in Figure 11, the computer device includes: a processor 1101, an RDMA network card 1102, a bus 1103, and a memory 1104. The processor 1101 and the RDMA network card 1102 are connected via the bus 1103. The RDMA network card includes: an RDMA engine 11021, temporary memory 11022, and a communication interface 11023.
[0542] Processor 1101 is primarily used for processing communication protocols and communication data, controlling the entire computer device, executing software programs, and processing data from those programs, such as supporting the computer device in performing the actions described in the embodiments. The computer device may include a baseband processor and a central processing unit (CPU). The baseband processor is primarily used for processing communication protocols and communication data, while the CPU is primarily used for controlling the entire computer device, executing software programs, and processing data from those programs. Processor 1101 in Figure 11 can integrate the functions of both a baseband processor and a CPU. Those skilled in the art will understand that the baseband processor and CPU can also be independent processors interconnected via technologies such as buses. Those skilled in the art will understand that a computer device may include multiple baseband processors to adapt to different network standards, and multiple CPUs to enhance its processing capabilities. The various components of the computer device can be connected via various buses. The baseband processor can also be described as a baseband processing circuit or a baseband processing chip. The CPU can also be described as a central processing circuit or a central processing chip. The function of processing communication protocols and communication data can be built into the processor or stored in memory as a software program, executed by the processor to implement the baseband processing function.
[0543] For example, processor 1101 may include any one or more processors such as CPU, GPU, MP, DPU, NPU or DSP.
[0544] For example, the computer device shown in Figure 11 can be understood as a computer device including a host and a network card, wherein the host includes a processor 1101 and a memory 1104. Accordingly, the memory 1104 can be referred to as the host's RAM.
[0545] The memory 1104 can also be referred to as the RAM of a computer device. It is primarily used to store software programs and data. The memory 1104 can exist independently, connected to the processor 1101. Optionally, the memory 1104 can be integrated with the processor 1101, for example, integrated within a single chip (i.e., the memory 1104 can also be referred to as an on-chip memory element). The memory 1104 is capable of storing program code that executes the technical solutions of the embodiments of this application, and its execution is controlled by the processor 1101. The various types of computer program code being executed can also be considered as drivers for the processor 1101.
[0546] For example, memory 1104 may include volatile memory, such as RAM. Memory 1104 may also include non-volatile memory, such as ROM, flash memory, HDD, or SSD.
[0547] Bus 1103 can be the Peripheral Component Interconnect Express (PCIe) standard, or other types of buses that enable connections between devices within a device. In addition to a data bus, bus 1103 can also include a power bus, a control bus, and a status signal bus.
[0548] The RDMA engine 11021 is primarily used for network protocol processing and data transmission processing. Network protocol processing mainly includes at least one of the following: packet parsing, routing, and error detection. Data transmission processing mainly includes at least one of the following: packet encapsulation and decapsulation, and data copying.
[0549] Temporary memory 11022 is mainly used to store temporary data, which can be data to be sent or data to be stored in memory 1104 after being received.
[0550] Communication interface 11023 is mainly used for communication with remote devices. For example, network card 1102 can communicate with other network cards through communication interface 11023.
[0551] For example, when the computer device is the receiving side, the RDMA engine 11021 can control the communication interface 11023 to first store the received data in temporary storage 1122, and then store the data in temporary storage 1122 in memory 1104.
[0552] Furthermore, there is no limitation on the number of the aforementioned processor 1101, memory 1104, RDMA engine 11021, temporary memory 11022, and communication interface 11023. For example, a computer device may include multiple processors 1101. As another example, an RDMA network card may include multiple communication interfaces 11023.
[0553] It should be noted that the computer device shown in Figure 11 can be used to implement the steps of the aforementioned method embodiments and achieve the corresponding technical effects. The specific implementation of the computer device shown in Figure 11 can be referred to the description in the aforementioned method embodiments, and will not be repeated here. For example, the computer device shown in Figure 11 can be the first computing device or the second computing device in Figure 1.
[0554] This application also provides a computing device cluster. The computing device cluster includes at least one computer device. The memory of one or more computer devices in the computing device cluster may store identical instructions for executing the aforementioned data communication method.
[0555] In some possible implementations, the memories of one or more computer devices in the computing device cluster may also each store a portion of the instructions for executing the data communication method. In other words, a combination of one or more computer devices can jointly execute the instructions for executing the data communication method.
[0556] For example, a computing device cluster includes multiple second computer devices as described above, which can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. In the computing device cluster, two second computer devices are connected via the network. Specifically, they are connected to the network through communication interfaces in each second computing device. In this type of possible implementation, the memory of one of the second computer devices stores instructions for performing the functions of the transceiver unit 801 shown in FIG. 8. Simultaneously, the memory of the other second computer device stores instructions for performing the functions of the processing unit 802 shown in FIG. 8.
[0557] This application also provides a computer storage medium storing one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the method steps of the embodiments shown in Figures 1 to 7.
[0558] This application also provides a computer program product that stores instructions that, when executed by a computer, cause the computer to perform the method steps of the embodiments shown in Figures 1 to 7.
[0559] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0560] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.
[0561] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0562] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0563] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A data communication method, characterized in that, The method is applied to a first network interface card (NIC) based on the Remote Direct Memory Access (RDMA) protocol, the first NIC being located in a first computer device containing a first processor, the method comprising: Receive first information sent by the first processor, the first information including: the storage address of the first data and the length of the first data; Based on the first queue, the QP sends a first message to the second network card in the second computer device. The first message includes a first payload and a first header. The first header includes: a connection identifier of the first source end, a connection identifier of the first destination end, and an identifier of the first data. The first payload is used to carry the first data. The state of the first QP is idle.
2. The method according to claim 1, characterized in that, Before sending the first message to the second network card in the second computer device based on the first QP, the method further includes: Identify the first QP that is idle in the first queue pair group QPG, and change the state of the first QP from idle to non-idle.
3. The method according to claim 1, characterized in that, The first information also includes first indication information, which is used to indicate that the first data is sent through the first QP.
4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: A first context is established for the first data, and the first context is used to maintain the transmission status of the first data.
5. The method according to any one of claims 1 to 4, characterized in that, The first QP includes: a send queue SQ and a completion queue CQ; the SQ is used to carry the first information; The step of sending the first message from the first queue to the second network card in the second computer device includes: The first data is obtained based on the SQ; Encapsulate the first data to obtain the first message; Send the first message to the second network interface card; The method further includes: Based on the CQ, a second message is sent to the first processor, the second message indicating that the first data transmission is complete.
6. The method according to claim 5, characterized in that, The second information includes the connection identifier of the first source end.
7. The method according to claim 5 or 6, characterized in that, After sending the second information to the first processor based on the CQ in the first QP, the method further includes: Change the state of the first QP from non-idle to idle.
8. The method according to claim 7, characterized in that, After changing the state of the first QP from non-idle to idle, the method further includes: The system receives third information sent by the first processor, the third information including: the storage address of the second data and the length of the second data, wherein the first data and the second data are generated by different application processes; Based on the first QP, a second message is sent. The second message includes a second payload and a second header. The second payload is used to carry the second data. The second header includes: a connection identifier of the second source end, a connection identifier of the second destination end, and an identifier of the second data.
9. The method according to any one of claims 1 to 8, characterized in that, The first QP belongs to the first queue pair group QPG, the first QPG is created by the first processor, and the first QPG is used to transfer data of multiple application processes on the first computer device based on RDMA.
10. A data communication method, characterized in that, The method is applied to a second network interface card (NIC) based on the Remote Direct Memory Access (RDMA) protocol, the second NIC being located in a second computer device containing a second memory, the method comprising: A first message is received from a first network interface card. The first message includes a first payload and first header information. The first header information includes: a connection identifier of a first source end, a connection identifier of a first destination end, and an identifier of the first data. The first payload is used to carry the first data. The reception of the first message does not use a queue pair (QP). The first data is stored in the second memory.
11. The method according to claim 10, characterized in that, The method further includes: A fourth message is sent to the second processor based on the completion queue CQ, the fourth message indicating that the first data reception is complete.
12. The method according to claim 11, characterized in that, The fourth piece of information includes the connection identifier of the first destination.
13. The method according to claim 11 or 12, characterized in that, The method further includes: Receive a second message, the second message including a second payload and second header information, the second header information including: a connection identifier of a second source end, a connection identifier of a second destination end, and an identifier of second data; wherein, the second payload is used to carry the second data, and the second data corresponds to a different application process than the first data; The second data is stored in the second memory; Based on the CQ, a fifth message is sent to the second processor, the fifth message indicating that the second data reception is complete.
14. The method according to any one of claims 10 to 13, characterized in that, The method further includes: A second context is established for the first data, which is used to maintain the reception status of the first data.
15. The method according to any one of claims 10 to 14, characterized in that, Before storing the first data in the second memory, the method further includes: The system receives a mapping table sent by the second processor. The mapping table records the mapping relationship between the connection identifier of the first destination and the target storage address. The target storage address is used to indicate the target storage space of the memory in the second computer device. The step of storing the first data in the second memory includes: The first data is stored in the target storage space based on the mapping table.
16. A computer device, characterized in that, The computer device includes: a first network interface card and a first processor; The first processor is configured to send first information to the first network interface card, the first information including: the storage address of the first data and the length of the first data; The first network interface card (NIC) is used to send a first message to the second NIC based on the first queue to the QP. The first message includes a first payload and first header information. The first header information includes: a connection identifier of the first source end, a connection identifier of the first destination end, and an identifier of the first data. The first payload is used to carry the first data. The state of the first QP is idle.
17. The computer device according to claim 16, characterized in that, The first QP includes: a send queue SQ and a completion queue CQ; the SQ is used to carry the first information; The first network interface card is specifically used to acquire the first data based on the SQ; The first network interface card is specifically used to encapsulate the first data to obtain the first message; The first network interface card (NIC) is specifically used to send the first message to the second NIC; The first network interface card is further configured to send second information to the first processor based on the CQ, the second information being used to indicate that the first data transmission is complete.
18. The computer device according to claim 16 or 17, characterized in that, The first processor or the first network interface card is further configured to change the state of the first QP from non-idle to idle; The first processor is further configured to send third information to the first network card, the third information including: the storage address of the second data and the length of the second data, wherein the first data and the second data are generated by different application processes; The first network interface card (NIC) is further configured to send a second message to the second NIC based on the first QP. The second message includes a second payload and second header information. The second header information includes: a connection identifier of the second source end, a connection identifier of the second destination end, and an identifier of the second data. The second payload is used to carry the second data.
19. The computer device according to any one of claims 16 to 18, characterized in that, The first QP belongs to the first queue pair group QPG, and the first QPG is used to transmit data of multiple application processes on the computer device based on RDMA; The first processor is also used to create the first QPG.
20. A computer device, characterized in that, The computer device includes: a second network interface card and a memory; The second network interface card (NIC) is used to receive a first message from the first NIC. The first message includes a first payload and first header information. The first header information includes: a connection identifier of a first source end, a connection identifier of a first destination end, and an identifier of the first data. The first payload is used to carry the first data. The reception of the first message does not use a queue pair (QP). The memory is used to store the first data received by the second network card.
21. The computer device according to claim 20, characterized in that, The computer device also includes a second processor; The second network interface card is also used to send the fourth information to the second processor based on the completion queue (CQ), the fourth information being used to indicate that the first data reception is complete.
22. The computer device according to claim 21, characterized in that, The second network interface card is also used to receive a second message, the second message including a second payload and second information, the second header information including: a connection identifier of the second source end, a connection identifier of the second destination end and an identifier of the second data, the second payload being used to carry the second data, the first data and the second data corresponding to different application processes on the computer device; The memory is also used to store the second data received by the second network card; The second network interface card is also used to send a fifth message to the second processor based on the CQ, the fifth message being used to indicate that the second data reception is complete.
23. A communication system, characterized in that, The communication system includes: a computer device as claimed in any one of claims 16 to 19, and a computer device as claimed in any one of claims 20 to 22.
24. A network interface card (NIC), characterized in that, The network interface card includes a memory and a processor; The memory stores code, and the processor is configured to execute the code. When the code is executed, the network interface card performs the method as described in any one of claims 1 to 15.
25. A computer device, characterized in that, The computer device includes a processor, memory, and a network interface card (NIC). The NIC is communicatively connected to the processor and to the memory. The NIC is configured to perform, based on instructions from the processor, the method described in any one of claims 1 to 9, or the method described in any one of claims 10 to 15, on the memory.
26. A computer storage medium, characterized in that, The computer storage medium stores a computer program that, when executed by the computer, causes the computer to perform the method described in any one of claims 1 to 15.
27. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1 to 15.
Citation Information
Patent Citations
In-chip connection management-based RDMA communication method
CN103248467A
Communication method and device for reliable connection
CN106921751A
Data sending method, sending terminal and receiving terminal
CN107257272A
Data transmission method based on RDMA communication, storage medium and equipment
CN117880216A
Remote direct memory access enabled network interface controller switchover and switchback support
US20040049774A1