A method, chip and device for data transmission

Abstract

A method for data transmission using RDMA technology, the method comprising: a network card of a source device obtaining a first notification message and a second notification message, wherein the first notification message is used to indicate that there is a first to-be-processed remote direct memory access (RDMA) request in a first queue of the source device, and the first queue is used to store requests of a first service application in the source device; the second notification message is used to indicate that there is a second to-be-processed RDMA request in a second queue of the source device, and the second queue is used to store requests of a second service application in the source device. The network card of the source device determines a processing order of the first queue and the second queue according to a service level of the first service application and a service level of the second service application, and sends the first to-be-processed RDMA request and the second to-be-processed RDMA request to a destination device respectively according to the processing order.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of communication, and in particular to a data transmission method, chip and device. BACKGROUND

[0002] With the development of computer technology, devices can use intelligent network interface cards (iNIC) to realize fast data access between devices through remote direct memory access (RDMA) technology. The remote direct memory access technology is based on network transmission queue pairs (QP) for data transmission, and accordingly, the receiving end device and the sending end device will establish a receive queue (RQ) and a send queue (SQ), respectively. Usually, the RDMA request to be processed of the service application in the source device is stored in the receive queue, then the processor notifies the network card to obtain the RDMA request to be processed, and sends the RDMA request to be processed to the destination device. However, the above process only executes the processing of the RDMA request to be processed according to the notification of the processor, and the network card cannot perceive the service level of the service application, cannot guarantee the importance of the service application, and causes the RDMA request to be processed of the important service application to be unable to be sent to the destination device in time, and the processing time delay of the important service application is high. Therefore, how to provide a method for reducing the processing time delay of the important service application becomes a technical problem to be solved. SUMMARY

[0003] The present application provides a data processing method, chip and device, which can perceive the service level of the service application and reduce the processing time delay of the important service application.

[0004] In a first aspect, a method for data transmission is provided. The method comprises: a network card of a source device obtaining a first notification message, wherein the first notification message is used to indicate that there is a first to-be-processed remote direct memory access (RDMA) request in a first queue of the source device, and the first queue is used to store an RDMA request of a first service application in a remote device; the network card obtaining a second notification message, wherein the second notification message is used to indicate that there is a second to-be-processed RDMA request in a second queue of the source device, and the second queue is used to store an RDMA request of a second service application in the remote device; then, the network card determines a processing order of the first queue and the second queue according to a service level of the first service application and a service level of the second service application, and sends the first to-be-processed RDMA request and the second to-be-processed RDMA request to a destination device according to the processing order. Through the above method, the network card can process the RDMA requests associated with the service applications in different orders according to the importance of the service applications, and meet the needs of service applications of different service levels. In a possible implementation, when the processing order of the first service application is earlier than the processing order of the second service application, the network card obtains the first to-be-processed RDMA request, and sends the first to-be-processed RDMA request to the destination device; then, the network card continues to obtain the second to-be-processed request, and sends the second to-be-processed RDMA request to the destination device. Through the above method, the network card can determine the processing order of the service applications according to the service levels, and then complete the transmission of the to-be-processed RDMA requests according to the processing order, thereby realizing the transmission of the to-be-processed RDMA requests according to the service levels of the service applications, so that the network card can perceive the service levels of the service applications, and ensure the processing delay of the to-be-processed RDMA requests according to the needs of the service levels.

[0005] In a possible implementation, when the processing order of the first service application is earlier than the processing order of the second service application, the network card obtains the first to-be-processed RDMA request, and sends the first to-be-processed RDMA request to the destination device; after sending the first to-be-processed request to the destination device, the network card obtains the second to-be-processed request, and sends the second to-be-processed request to the destination device. Through the above process, it can be known that the network card can determine the processing order according to the service levels of the service applications, and then send the to-be-processed RDMA requests of the multiple queues to the destination device according to the determined processing order, realize the process of sending the to-be-processed RDMA requests according to the service levels of the service applications, ensure that the to-be-processed RDMA requests of the service applications of high service levels are sent to the destination device in priority, and reduce the processing delay of the to-be-processed RDMA requests of the service applications of high service levels.

[0006] In another possible implementation, the network card further comprises a memory, and before the network card sends the RDMA request to be processed in the first queue to the destination device, the network card further determines a storage manner of the context of the first queue in the memory according to a preset rule. The context of the first queue is used to indicate the identifier of the destination device and the identifier of the queue associated with the first queue in the destination device. Then, the network card acquires the context of the first queue and stores the context of the first queue in the memory of the network card according to the storage manner. Through the above method, the network card can save the context of the first queue according to the preset rule, and then store the context of the first queue to the memory of the network card according to the determined storage manner, so that the network card can implement the transmission process of the RDMA request to be processed of different service applications according to the processing order.

[0007] In another possible implementation, the network card determines the storage manner of the context of the first queue in the memory according to the operation type of the RDMA request to be processed in the first queue, wherein the operation type comprises a persistent collective operation. Through the above process, the network card can determine the storage manner of the context of the queue to which the RDMA request to be processed belongs according to the operation type of the RDMA request to be processed, and then store the context of the queue according to the storage manner, so as to reduce the time delay caused by the frequent access of the network card to the memory of the device, and further reduce the processing time delay of the service application in transmitting the RDMA request to be processed.

[0008] In another possible implementation, the network card compares the size of the data packet in the RDMA request to be processed in the first queue with the size of the first threshold value, and determines the storage manner of the context of the first queue in the memory according to the comparison result. As known from the above description, the network card can determine the storage manner according to the size of the data packet of the RDMA request to be processed. If the data packet is smaller, the network card must frequently acquire the context of the queue in the memory during the process of sending each data packet. The context of the queue can be stored in the memory of the network card, so as to reduce the process of frequent access of the network card to the memory, and further reduce the time delay of data transmission.

[0009] In another possible implementation, the network card compares the usage frequency of the first queue with a second threshold value, and the first queue comprises a plurality of work queue elements (WQE), and each WQE is used to store an RDMA request to be processed, wherein the usage frequency is used to indicate the number of the elements in the first queue storing the RDMA request to be processed within a first time threshold value. Then, the network card determines the storage manner of the context of the first queue in the memory according to the comparison result. As known from the above description, the network card can adopt different storage manners of the context of the queue according to the usage frequency of the queue, and save the context of the frequently used queue in the memory in time, so as to reduce the processing time delay of the network card in frequently accessing the memory to acquire the context of the queue, and further reduce the processing time delay of the RDMA request to be processed.

[0010] In another possible implementation, the network card determines the storage manner of the context of the first queue in the memory according to the service level of the first service application. The network card can take different storage manners of the queue context according to the service level of the service application, and the corresponding queue context of the service application with a high service level is stored for a long time, so as to reduce the processing time of the request of the service application with the high service level.

[0011] In another possible implementation, the first queue and the context of the first queue are stored in the memory of the source device, and the network card stores the context of the first queue in the memory of the network card according to the storage manner after obtaining the context of the first queue from the memory, including: the network card stores the context of the first queue in the first storage area of the memory of the network card, and retains the context of the first queue in the first storage area within the first time threshold. That is, the network card can divide the storage area in the memory according to the determined storage manner, and the storage area is used to store the queue context within a specified period of time, so as to reduce the time of reading the queue context of the queue by the network card next time, and improve the efficiency of processing the RDMA request.

[0012] In another possible implementation, the second queue and the context of the second queue are stored in the memory of the source device, and the network card stores the context of the second queue in the memory of the network card according to the storage manner after obtaining the context of the second queue from the memory, including: the context of the second queue is stored in the second storage area of the memory of the network card, and the context of the second queue is deleted from the second storage area when the sending of the second to-be-processed RDMA request is completed. As described above, the network card can delete the queue context that is not important from the memory of the network card after the queue context is used according to the determined storage manner, so as to avoid the waste of the priority memory resource caused by blind caching on the network card.

[0013] In another possible implementation, the network card obtains the context of the first queue or the context of the second queue from the memory of the source device through direct memory access (DMA). The network card can quickly obtain the first queue context and the second queue context by using the DMA technology, so as to avoid the time delay problem caused by the participation of the processor of the device in the data transmission process, and further improve the processing efficiency of the to-be-processed RDMA request.

[0014] In another possible implementation, when the remaining space of the first storage area is less than the preset threshold, the network card stores the context of the third queue in the second storage area, and the context of the third queue is kept in the second storage area within the first time threshold. The context of the third queue is used to indicate the identifier of the destination device and the identifier of the queue associated with the third queue in the destination device. The third queue is used to store the pending RDMA request of the third service application. Through the above method, in addition to the way of expanding the first storage area to ensure that the first storage area has sufficient storage space, the network card can also directly store the queue context associated with the service application of the specified service level in the storage area other than the first storage area, for example, the second storage area, and ensure that the storage duration of the queue context is consistent with the storage duration of the data in the first storage area, thereby ensuring that the queue context of the service application of the specified service level can be stored in the storage of the network card for a long time, avoiding the problem of long processing delay caused by frequent access of the network card to the memory of the device, and reducing the processing delay of the pending RDMA request of the service application of the specified service level.

[0015] In another possible implementation, when the remaining space of the first storage area is less than the preset threshold, the network card stores the context of the third queue in the second storage area, and the context of the third queue is kept in the second storage area within the first time threshold. The context of the third queue is used to indicate the identifier of the destination device and the identifier of the queue associated with the third queue in the destination device. The third queue is used to store the pending RDMA request of the third service application. Through the above method, in addition to the way of expanding the first storage area to ensure that the first storage area has sufficient storage space, the network card can also directly store the queue context associated with the service application of the specified service level in the storage area other than the first storage area, for example, the second storage area, and ensure that the storage duration of the queue context is consistent with the storage duration of the data in the first storage area, thereby ensuring that the queue context of the service application of the specified service level can be stored in the storage of the network card for a long time, avoiding the problem of long processing delay caused by frequent access of the network card to the memory of the device, and reducing the processing delay of the pending RDMA request of the service application of the specified service level.

[0016] In another possible implementation, when the remaining space of the second storage area is less than the preset threshold, the network card stores the context of the second queue in the second storage area of the memory of the network card, and deletes the context of the fourth queue in the second storage area. The context of the fourth queue is used to indicate the identifier of the destination device and the identifier of the queue associated with the fourth queue in the destination device. The fourth queue is used to store the pending RDMA request of the fourth service application. Through the above method, the network card can replace the previously stored queue context with the currently used queue context, thereby ensuring that the processing delay of the pending RDMA request of the service application of the service application of the specified service level meets the demand of the service level.

[0017] In another possible implementation, when the remaining space of the second storage area is less than the preset threshold, the network card stores the context of the second queue to the second storage area of the memory of the network card, and deletes the context of the fifth queue in the second storage area, wherein the context of the fifth queue is used to indicate the identity of the destination device and the identity of the queue associated with the fifth queue in the destination device, the fifth queue is used to store the RDMA request to be processed of the fifth service application, and the storage level of the context of the fifth queue is lower than the context of the second queue. Through the above method, the network card can replace the queue context with a lower storage level with the queue context corresponding to the service application with a specified service level, save storage space, and preferentially guarantee the processing of the request of the more important service application.

[0018] In another possible implementation, when the remaining space of the second storage area is less than the preset threshold, the network card stores the context of the second queue to the second storage area of the memory of the network card, and deletes the context of the sixth queue in the second storage area, wherein the context of the sixth queue is used to indicate the identity of the destination device and the identity of the queue associated with the sixth queue in the destination device, the sixth queue is used to store the RDMA request to be processed of the sixth service application, and the storage level of the context of the sixth queue is the lowest among the contexts of all the queues in the second storage area. Through the above method, the network card can replace the queue context with the lowest storage level with the queue context corresponding to the service application with a specified service level, save storage space, and preferentially guarantee the processing of the request of the more important service application.

[0019] In another possible implementation, the network card judges whether the deleted context is dirty data, and if yes, the network card stores the deleted context in the context cache area of the corresponding queue in the memory, and if not, the network card directly deletes the deleted context. Through the above method, the network card can synchronize the queue context in the memory and the queue context in the memory.

[0020] In another possible implementation, the network card acquires the first RDMA request to be processed from the memory of the source device, and stores the first RDMA request to be processed to the memory of the network card; then, the network card sends the first RDMA request to be processed to the destination device; after sending the first request to be processed to the destination device, the network card acquires the second request to be processed, and sends the second request to be processed to the destination device, including: the network card acquires the second RDMA request to be processed from the memory of the source device, stores the second RDMA request to be processed to the memory of the network card, and sends the second RDMA request to be processed to the destination device. The network card determines the processing order according to the service level of the service application, and completes the data transmission of the RDMA request to be processed of different queues according to the determined processing order, thereby guaranteeing the processing delay of the service application with a specified service level.

[0021] In another possible implementation manner, the network card is an intelligent network card supporting the RDMA function.

[0022] In another possible implementation manner, the network card comprises a doorbell register comprising notification messages associated with each queue, wherein the first notification message comprises the service level of the first service application, and the second notification message comprises the service level of the second service application. When the network card is notified of the RDMA request to be processed, the processor of the source device can notify the network card of the service level of the service application, so that the network card can determine the processing order of the RDMA request to be processed according to the service level, and ensure the processing delay of the RDMA request to be processed of the service application of the specified service level according to the determined processing order.

[0023] In another possible implementation manner, the first queue comprises any one of a send queue (SQ), a receive queue (RQ) or a completion queue (CQ), wherein the send queue and the receive queue form a queue pair, and the first queue context comprises a queue pair context or a completion queue context. As can be seen from the above, the processing delay of the RDMA request to be processed of the specified service level can be reduced in the processing of the SQ, the RQ and the CQ according to the method provided in the present application.

[0024] In a second aspect, a network card is provided, which comprises various modules for executing the data transmission method in the first aspect or any possible implementation manner of the first aspect.

[0025] In a third aspect, a chip is provided, which comprises a processor and a communication interface for communicating with the processor of the device in which the chip is located, and the processor of the chip is configured to implement the operation steps of the method executed by the network card in the first aspect or any possible implementation manner of the first aspect.

[0026] In a fourth aspect, a device is provided, which comprises a first processor, a memory and a network card, wherein the first processor is configured to run a plurality of service applications, each service application being associated with a queue, and the memory is configured to store remote direct memory access (RDMA) requests to be processed in the queues associated with the plurality of service applications. The network card is configured to implement the operation steps of the method executed by the network card in the first aspect or any possible implementation manner of the first aspect.

[0027] In a fifth aspect, a computer readable storage medium is provided, which stores instructions, and when the instructions are run on a computer, the computer is caused to execute the operation steps of the method in the first aspect or any possible implementation manner of the first aspect.

[0028] In a sixth aspect, a computer program product containing instructions which, when executed on a computer, cause the computer to perform the operational steps of the method of the first aspect or any of the possible implementation forms of the first aspect is provided.

[0029] On the basis of the implementation forms of the aspects described above, the present application can be further combined to provide more implementation forms. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 Structure schematic diagram of a data processing system 100 provided by an embodiment of the present application

[0031] Figure 2 Structure schematic diagram of another data processing system provided by an embodiment of the present application

[0032] Figure 3 Structure schematic diagram of a source terminal device 20 provided by an embodiment of the present application

[0033] Figure 4 Flowchart of a QP-based communication between devices provided by an embodiment of the present application

[0034] Figure 5 Another flowchart of a QP-based communication between devices provided by an embodiment of the present application

[0035] Figure 6 Flowchart of a data processing method provided by the present application

[0036] Figure 7 Flowchart of a QPC caching method provided by the present application

[0037] Figure 8 Another flowchart of a QPC caching method provided by the present application

[0038] Figure 9 Specific example diagram of a data processing method provided by the present application

[0039] Figure 10 Flowchart of a method for modifying a QPC using application software provided by the present embodiment

[0040] Figure 11 Flowchart of a method for deleting a QPC using application software provided by the present embodiment

[0041] Figure 12 Schematic diagram of a data transmission apparatus provided by the present application

[0042] Figure 13 Schematic diagram of a device provided by an embodiment of the present application DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0044] Figure 1 A structural schematic diagram of a data processing system 100 provided by an embodiment of the present application is shown in the figure. The system 100 includes devices 11-14 and a network 15. The devices communicate with each other through the network 15. The network 15 supports a protocol for remote direct memory access (RDMA) technology, such as the InfiniBand (IB) protocol, the RDMA over Converged Ethernet (RoCE) protocol, and the internet Wide Area RDMA Protocol (iWARP). In the implementation process, one or more switches and / or routers can be used to implement the communication processing between multiple nodes. For ease of description, the devices 11-14 are described as servers, and the network 15 is described as a network. Figure 1 In the data processing system 100 shown in the figure, the device that sends the RDMA request to be processed is referred to as a source device, and the device that receives the RDMA request to be processed is referred to as a destination device.

[0045] The devices 11-14 can be computing devices (e.g., servers), storage devices (e.g., storage arrays), or other devices that support RDMA technology. It should be understood that the number of devices in the system 100 does not limit the present application, Figure 1 The system is described by way of example only.

[0046] The communication method of the source device and the destination device based on the RDMA technology will be described below. Figure 2 The communication method of the source device and the destination device based on the RDMA technology will be described below.

[0047] Figure 2 A structural schematic diagram of another data processing system provided by an embodiment of the present application is shown in the figure, Figure 2 The source device 20 and the destination device 30 that use the RDMA technology to perform data transmission are described by way of example. The source device 20 or the destination device 30 can be any one of the devices in Figure 1

[0048] ​The source device 20 includes a first processor 21, a memory 22 and a network card 23, wherein the first processor 21, the memory 22 and the network card 23 are connected through a bus, for example, a Peripheral Component Interconnect Express (PCIe). The bus can also be a bus of other types for realizing the connection between devices in the apparatus. In addition, the bus can include a data bus, a power supply bus, a control bus and a state signal bus, etc. in addition to the data bus.

[0049] The first processor 21 can be a central processing unit (CPU). The first processor 21 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. For ease of description, the following embodiments take the first processor 21 as a CPU for example.

[0050] The network card 23 can also be referred to as a host channel adapter (HCA). The network card 23 is an intelligent network card supporting the RDMA technology. The RDMA technology reduces the process of the processor participating in data transmission.

[0051] The source device 20 can directly send the data stored in the memory to the destination device 30 through the network 15, and the data is received by the network card on the destination device 30 and stored in the memory.

[0052] As a possible embodiment, the hardware structures of the source device 20 and the destination device 30 can be the same or different. For ease of description, the following embodiments take the hardware structures of the source device 20 and the destination device 30 as the same for example.

[0053] Next, the hardware structure of the source device 20 will be introduced in detail. Figure 3 The source device 20 further includes a bus interface 24, a memory controller 25, a network card controller 26, a host channel adapter (HCA) 27, a direct memory access (DMA) controller 28 and a power supply 29. Figure 3 A structure schematic diagram of the source device 20 provided by the embodiments of the present application is shown in the figure.

[0054] The first processor 21 runs multiple business applications, such as business application 1 and business application 2. Each business application can generate work requests (WRs), including RDMA write requests, RDMA read requests, and RDMA send / receive requests. These business applications include database applications, high-performance computing (HPC) applications, etc.

[0055] Optionally, business applications can also generate work requests by calling interfaces provided by the middleware layer. For example, in high-performance computing scenarios, business applications can generate different parallel computing requests, such as non-blocking communication requests and set operation requests, by calling the application programming interfaces (APIs) of various parallel computing technologies provided by the message passing interface (MPI). In storage scenarios, business applications can generate data transmission requests by calling the Exploitation Network (xNet) communication component.

[0056] Memory 22 can store work requests generated by the business application, data that the business application needs to transmit, and data structures needed during RDMA transmission. For example, work requests generated by the business application can be stored in memory using interconnect context memory (ICM), which can be a storage area in memory. Specifically, RDMA technology can utilize queue pairs (QPs) (e.g., ...). Figure 1 The queue pairs 1 and 2 shown store work requests from business applications. Additionally, the interconnect context memory is also used to store queue pair contexts (QPCs) (e.g., Figure 1 The queue pair context 1 and queue pair context 2 are shown, along with the completion queue (CQ), completion queue context (CQC), and memory key context (MKC).

[0057] Wherein, the QP is also called a network transmission queue pair, when the source device and the destination device communicate using the RDMA technology, the first processor 21 of the source device allocates at least one queue pair for each service application. Each queue pair includes a send queue (SQ) and a receive queue (RQ). Wherein, the SQ is used to store the work request sent by the service program to the destination device, and the RQ is used to store the work request received by the service program sent by the destination device. The SQ of each data sender is associated with the RQ of each data receiver. Each SQ or each RQ includes a plurality of entries respectively, each entry can also be called a work queue entry (WQE), and each WQE is used to store an RDMA request generated by the service application, also called a work request (WR).

[0058] The source device 20 and the destination device 30 can communicate through the queue pair, specifically, Figure 4 is a flow diagram of the inter-device communication based on the QP provided by the embodiment, as shown, the source device 20 and the destination device 30 establish a communication connection through QP1 and QP11 in the initialization stage, and then realize the communication of the service program deployed in the source device based on the QP1 of the source device and the QP11 of the destination device. Wherein, the QP1 includes SQ1 and RQ1, and the QP11 includes SQ11 and RQ11, when the SQ1 is associated with the RQ11, the RDMA sending request generated by the service application running on the source device 20 can be stored in a WQE of the SQ1, at the same time, the first processor of the destination device 30 also generates an RDMA receiving request and stores it in a WQE of the RQ11. Similar to the SQ1 and the RQ11, when the RQ1 is associated with the SQ11, the RDMA receiving request generated by the service application running on the source device 20 can be stored in a WQE of the SQ1, at the same time, the first processor of the destination device 30 also generates an RDMA sending request and stores it in a WQE of the RQ11.

[0059] Further, the head pointer and the tail pointer can be used to indicate the storage condition of the WQE in the queue. For example, as Figure 4As shown, the head pointer of SQ1 points to the address of WQE2, and the tail pointer points to the address of WQE1, at this time, only WQE2 in SQ1 stores the RDMA request. When the service application running on device 20 generates an RDMA sending request, first processor 21 will place the request in WQE3 of SQ1, at this time, the head pointer of SQ1 will move down by one storage unit corresponding to the storage location and point to WQE3; when the network card takes out the RDMA request stored in WQE2 from SQ1, the tail pointer of SQ will move down by one storage unit corresponding to the storage location and point to WQE2.

[0060] Each QP corresponds to a QPC, which is used to store the configuration data of the QP, and Table 1 is an example of the data format of the QPC. It should be noted that Table 1 is only an example and does not constitute a limitation on the data format of the QPC, and the data format of the QPC can also include other fields.

[0061] Table 1, QPC data format

[0062] Field Meaning QPN Number of QP, each QP has a unique number QPN of destination device Number of another QP in QP pair SQ address Address of storage space storing SQ SQ address index Address of storage unit pointed by tail pointer of SQ

[0063] As shown in Table 2, the network card can use the SQ address in the QPC to locate the address of the storage space of the SQ in which the to-be-processed RDMA request is stored, and then use the SQ address index to locate the address of the to-be-processed RDMA request.

[0064] The CQ includes a plurality of completion queue elements (CQE), which are used to store the notification returned to the service application after the network card executes the work request or after the network card executes the work request and receives the response from the destination device, including a flag indicating whether the request is correctly executed and an error code. Similar to the QP, each CQ also corresponds to a CQC, which stores the attributes related to the CQ.

[0065] Before the source device and the destination device perform RDMA communication, the first processor 21 will allocate a storage region in the memory for the service application of the source device, which is used to store the data to be sent and received in the work request sent by the service application, and is referred to as a memory region (MR). The memory region can store the authentication identifiers of the source device and the destination device, for example, a local key (L_KEY) and a remote key (R_KEY), the local key is used to save the access permission to the memory region of the source device, and the remote key is used to save the access permission to the memory region of the destination device. Figure 5 is another flowchart of the QP-based communication between devices provided by the embodiments of the present application, as shown in Figure 5As shown, the network card of the source device 20 needs to use the local key when accessing the registration area in the device, and the source device needs to use the remote key when accessing the registration area in the destination device 30. Each MR also corresponds to an MKC, which is used to store the authentication identifier and address conversion relationship of the MR, including the local key, the remote key, and a mapping table for mapping the logical address (LBA) to the physical address (PBA). The mapping table can obtain the physical address of the data to be transmitted in the memory according to the logical address of the data to be transmitted stored in the WQE.

[0066] The memory 22 can include read-only memory and random access memory, and provide instructions and data for the first processor 21. The memory 22 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0067] The network card 23 is a network card supporting the RDMA function, which supports a custom or standard RDMA protocol network port, such as at least one of the IB protocol, the RoCE protocol, and the iWAPP

[0068] It should be noted that at least one network card can be included in one device, and each network card can independently process work requests, such as Figure 3As shown, the network card 23 includes a second processor 231, a memory 232 and a port 233. For simplicity, elements of the network card 23 that are not essential to understanding the present application are omitted.

[0069] Generally, the first processor 21 allocates an address range in the memory space to the network card, and when there is a work request to be executed in the QP, the first processor 21 notifies the network card by changing the identity of the corresponding QP in the doorbell register of the network card 23. In response to the doorbell, the computing unit 2311 of the network card acquires the QPC of the QP indicated by the doorbell, and takes out and executes the work request according to the SQ address and the SQ address index in the QPC, for example, in the case of an RDMA read request.

[0070] After the work request is completed, the computing unit 2311 generates a CQE and stores it in the CQ to notify the service application of the completion of the request. Similar to the QP, the computing unit also needs to use the CQC corresponding to the CQ when generating the CQE.

[0071] As described above, when processing the work request of the service application, the computing unit 2311 needs to use the QPC, the MKC and the CQC. In order to improve the processing performance of the network card, the QPC, the MKC and the CQC can be stored in the memory 232 of the network card.

[0072] Due to the limitation of the memory size, the memory of the network card generally cannot save all the QPC, the MKC and the CQC used by the service application. Therefore, in the case where the storage space of the memory of the network card is insufficient, the QPC, the MKC and the CQC of important service applications can be stored preferentially.

[0073] The resident cache space 2322 and the non-resident cache space 2321 can be allocated in the memory space of the network card, and when the memory space is insufficient, the QPC stored in the resident cache will not be replaced within a first time threshold, and the QPC stored in the non-resident cache will be replaced. The storage control module 4312 can be used to manage the memory space and complete the replacement of the QPC.

[0074] Correspondingly, taking the QPC as an example, the storage type of the QPC can also be divided into resident and non-resident according to the storage mode of the QPC. When the memory space is insufficient, the resident QPC is not allowed to be replaced by a new QPC within a first time threshold, and the non-resident QPC can be replaced by a new QPC within the first time threshold. The replacement method includes random replacement or using the least recently used (LRU) replacement strategy.

[0075] Optionally, the storage type of the QPC can also be identified by a storage level, and the QPC of each storage type can correspond to at least one storage level, for example, the QPC of the storage levels 4 and 5 are designated as the resident QPC. For the non-resident QPC, the order of the QPC to be replaced can be determined according to the storage level, and the QPC with a lower storage level is preferentially replaced. The storage level of the QPC can be directly identified by using the existing field in the RDMA transmission protocol, for example, the service level SL field in the IB protocol. Optionally, the storage level of the QPC can also be identified by using the newly added custom field in the RDMA transmission protocol.

[0076] It should be noted that various units constituting the network card 23 can be implemented as hardware circuits or software processes running on programmable processors, or as a combination of hardware and software implemented elements.

[0077] Based on Figure 2 The system for communication between the source device and the destination device based on the RDMA technology shown in the figure, the embodiment of the present application provides a data processing method, which preferentially processes the RDMA request of the service application of the specified service level, and can store the QPC corresponding to the service application of the specified service level in the memory of the network card, thereby improving the speed of the work request of the transmission service application.

[0078] The structure of the data processing system provided by the embodiment of the present application is described in detail above, and the method of how to preferentially process the RDMA request of the service application of the specified service level will be described below with the Figures 1 to 5 Figures 6 to 8 Taking the RDMA read request as an example, the method of how to preferentially process the RDMA request of the service application of the specified service level is described.

[0079] Figure 6 is a flowchart of the data processing method provided by the present application, and the specific steps are as follows:

[0080] S601, the network card of the source device obtains the service level of the service application.

[0081] The work request of the service application will be put into the queue allocated by the first processor, for example, the work request can be a to-be-processed RDMA write request, which will be put into the SQ allocated by the first processor. The first processor interacts with the doorbell module of the network card through the software and hardware interface, and notifies the network card that there is a to-be-processed RDMA write request in the SQ. The first processor can also send the service level of the service application to the network card at the same time of sending the notification information to the network card. The service level of the service application can be determined according to the performance requirement of the service application, for example, the bandwidth requirement of the service application or the time delay sensitivity of the service application.

[0082] ​When there are RDMA write requests to be processed in at least two SQ queues at the same time, the network card determines the processing order of the RDMA requests to be processed according to the service level of the service application after obtaining the service level of the service application. For example, the first SQ stores a first RDMA write request to be processed generated by a first service application, and the second SQ queue stores a second RDMA write request to be processed generated by a second service application, wherein the service level of the first service application is higher than the service level of the second service application, and the network card will preferentially process the first RDMA request to be processed in the first SQ.

[0083] S602, the network card of the source end obtains the QPC according to the service level.

[0084] The network card obtains the QPC corresponding to the SQ from the memory according to the QPN of the first RDMA request to be processed generated by the first service application according to the service level of the service application. According to the storage mode of the QPC, the network card saves the QPC in different types of storage spaces. Specifically, the QPC with a storage type of permanent is stored in a permanent cache, and the QPC with a storage type of non-permanent is stored in a non-permanent cache.

[0085] In a possible embodiment, the network card retrieves whether the QPC corresponding to the SQ exists in the memory of the network card according to the QPN, as shown in Figure 3 The network card directly obtains the QPC from the memory when the QPC exists in the memory. When the QPC does not exist in the memory, the network card obtains the QPC corresponding to the SQ from the memory according to the QPN.

[0086] Optionally, the network card can obtain the QPC from the memory of the source end device through direct memory access (DMA).

[0087] Optionally, if the storage type of the QPC is permanent and is saved in the non-permanent cache, the network card can further determine whether there is remaining space in the permanent cache, as shown in Figure 3 The cache controller 2312 can be shown. If there is remaining space in the permanent cache, the network card migrates the QPC from the non-permanent cache to the permanent cache and releases the original storage space.

[0088] The computing unit in the network card can determine the storage mode of the QPC according to at least one of the following rules.

[0089] Rule one: determine the storage mode of QPC according to the service level of the service application associated with the SQ being used. The storage type of the QPC corresponding to the SQ used by the service application of the specified service level can be set as resident, and saved in the resident cache, and the storage type of the QPC corresponding to the rest of the level can be set as non-resident, and saved in the non-resident cache. Alternatively, the service level can be used as the storage level of the QPC.

[0090] Rule two: the network card can obtain the transmission characteristics of all the SQs being used from the first processor, such as the operation type of the work request to be processed in the SQ, the size of the data packet in the RDMA request to be processed in the SQ, or the use frequency of the SQ. The storage type of the QPC can be determined according to at least one transmission characteristic, including:

[0091] 1) determine the storage mode of QPC according to the operation type of the first to-be-processed RDMA write request in the SQ. The storage type of the QPC of all SQs used by one operation type can be specified, for example, in the high-performance computing (HPC) scenario, message passing interface (MPI) is required to be used for message passing between different processes. In the standard of MPI, a persistent collective operation application interface is defined, which fixes the parameters of the sending and receiving interfaces, including the number, type, and length of the parameters. Compared with the traditional operation type, the persistent collective operation uses fixed QP, CP, and MK for each transmission. Therefore, the storage type of the QPC of all SQs used by the persistent collective operation can be set as resident, and saved in the resident cache. Alternatively, the storage level of the QPC of all SQs used by one operation type can also be specified.

[0092] 2) determine the storage mode of QPC according to the size of the data packet in the first to-be-processed RDMA request in the SQ: the storage type of the QPC corresponding to the SQ used by the data packet smaller than the first threshold can be set as resident, and saved in the resident cache, and the storage type of the rest of the QPC can be set as non-resident, and saved in the non-resident cache. For example, the storage type of the QPC corresponding to the data packet smaller than 128B can be set as resident, and stored in the resident cache. If the data packet is smaller, the network card must frequently obtain the queue context in the memory during the process of sending each data packet, and the queue context can be stored in the network card memory to reduce the process of the network card frequently accessing the memory, and further reduce the time delay of data transmission. Alternatively, the storage level of the QPC can also be set according to the size of the data packet, and the storage level of the QPC corresponding to the SQ of the smaller data packet is higher.

[0093] 3) SQ Usage Frequency: SQ usage frequency refers to the number of pending RDMA requests stored in the SQ within the second time threshold. The storage type of the QPC corresponding to the SQ with a usage frequency greater than the second threshold is set to resident and stored in the resident cache. The storage type of the remaining QPCs is set to non-resident and stored in the non-resident cache. Optionally, the storage level of the QPC can also be set according to the QP's usage frequency; the higher the usage frequency of the SQ, the higher the storage level of the QPC corresponding to that SQ.

[0094] The network interface card (NIC) can set the size of its resident cache storage space and non-resident cache storage space in memory according to its own communication capabilities (e.g., the size of RAM on the NIC chip) and the needs of business application transmission tasks (e.g., the bandwidth required by the transmission task or the sensitivity of the transmission task to latency).

[0095] As one possible implementation, a network management device can be used before the network card obtains the storage method of the QPC. Figure 1 (Not shown in the image) Configure the network interface card (NIC) on the device performing data transmission. The network management device can be a separate server connected to the same network as the source device, or it can be the source device itself. The network management device can obtain the communication capabilities of the NICs of all devices in the network, and can configure the size of the resident and non-resident caches on each NIC according to the communication capabilities of the NICs and the needs of business application transmission tasks.

[0096] S603, The source network card obtains the working request.

[0097] The source network card locates the first pending RDMA write request based on the SQ address and SQ address index obtained from the QPC, and then obtains the request.

[0098] Alternatively, the network card can obtain the first pending RDMA request from the memory of the source device via Direct Memory Access DMA.

[0099] Optionally, after the network card obtains the first pending RDMA request, it can also save the QP in the memory so that if the first pending RDMA request fails to be sent, the network card can directly reread the working request from the memory.

[0100] S604. The network interface card of the source device sends a work request to the network interface card of the destination device.

[0101] After the network card receives the first pending RDMA request, it encapsulates the request into the corresponding data format according to the specific protocol supported by the network card and sends it to the destination device.

[0102] Optionally, the network card of the source device sends the service level of the business application and the work request to the network card of the destination device.

[0103] S605, the network card of the source device returns the execution result of the work request.

[0104] After the sending is completed, the network card records the execution result of the SQ in the CQ. When the network card of the source device and the network card of the destination device are in a reliable connection, the network card of the source device needs to wait for the network card of the destination device to return the receiving response before generating an execution result; when the network card of the source device and the network card of the destination device are in an unreliable connection, the network card of the source device can immediately generate an execution result. The network card needs to use the CQC when putting the execution result into the CQ.

[0105] The network card can use a method similar to the processing of the QPC to obtain and store the CQC.

[0106] It should be noted that the data transmission method of the destination device can be processed according to the method of the source device, or can be processed according to the method thereof. The method of processing data of the destination is not limited.

[0107] It can be known from the description of the above process that the data processing method provided by the embodiment of the application can preferentially transmit the work request of the business application of the specified service level according to the service level of the business application, and can store the QPC to the memory of the network card according to the determined storage mode according to the transmission characteristics of the QP and the service level of the business application, so that the network card can preferentially guarantee the transmission efficiency of the RDMA request of the business application of the specified service level.

[0108] Figure 7 is a flowchart of a QPC caching method provided by the application, as shown in the figure, and the specific method is as follows:

[0109] S701, the network card judges whether the QPC needs to be cached. If not, the current QPC does not need to be cached; if yes, continue to execute S702.

[0110] S702, the network card judges whether the QPC needs to be stored in the resident cache according to the method in S603.

[0111] If yes, execute S705 to judge whether there is remaining storage space in the resident cache. When there is no remaining space, execute S706 to set the storage type of the current QPC as non-resident, and continue to execute S703; when there is remaining space, execute S707 to save the current QPC in the resident cache.

[0112] If not, continue to execute S703

[0113] S703, the network card judges whether there is remaining storage space in the non-resident space.

[0114] When there is remaining storage space, S709 is performed to save the QPC in the QPC storage remaining space.

[0115] When there is no remaining storage space, the network card performs S704.

[0116] S704, the network card uses the current QPC to replace the existing QPC, and the replacement strategy can use any of the following strategies.

[0117] Strategy one: use the current QPC to replace the existing QPC in the cache whose storage type is non-resident.

[0118] Strategy two: use the current QPC to replace the existing QPC in the cache whose storage level is less than the current QPC.

[0119] Strategy three: use the current QPC to replace the existing QPC in the cache whose storage level is the lowest among all QPCs in the current cache.

[0120] S710, the network card judges whether the replaced QPC is dirty data.

[0121] If yes, the network card performs S711 to re-write the dirty data into the corresponding QPC in the memory.

[0122] If not, the network card performs S712 to release the replaced QPC.

[0123] As can be seen from the above description of the process, the network card can preferentially guarantee the storage mode of the QPC of the specified service level. When the storage space of the first storage area is insufficient, the first storage area can be expanded by adding a new storage area, so that the QPC of the business application of the specified service level can be stored according to the determined storage mode, thereby guaranteeing the transmission delay of the RDMA request to be processed by the business application of the service level.

[0124] Figure 8 is another flowchart of a QPC caching method provided by the present application, in which Figure 8 In the method shown, the network card only needs to judge the storage type of the QPC to determine whether to save the QPC in the memory for a long time, without specifying a specific storage area. As shown in the figure, the specific method is as follows:

[0125] S801, the network card judges whether the QPC needs to be cached. If yes, the current QPC does not need to be cached, and step S604 is directly performed. If not, S802 is continued.

[0126] S802, the network card judges whether there is remaining cache space in the memory.

[0127] When there is, the cache controller performs S804, directly storing the QPC in the remaining cache space.

[0128] When there is not, S805 is performed to judge whether the storage type of the current QPC is permanent and the number of entries of the QPC with the storage type of permanent already existing in the cache is greater than or equal to a third threshold value. When the result of the judgment is yes, the network card performs S806 to set the storage type of the current QPC to non-permanent and continues to perform S803; when the result of the judgment is no, the cache controller directly performs S803.

[0129] S803, the network card replaces the QPC already existing in the cache with the current QPC. The replacement strategy can use any of the following strategies.

[0130] Strategy one: the QPC already existing in the cache with the storage type of non-permanent is replaced with the current QPC.

[0131] Strategy two: the QPC already existing in the cache with the storage level less than that of the current QPC is replaced with the current QPC.

[0132] Strategy three: the QPC already existing in the cache with the storage level being the lowest among all the QPCs in the current cache is replaced with the current QPC.

[0133] S807, the network card judges whether the replaced QPC is dirty data.

[0134] If yes, the network card performs S808 to re-write the dirty data into the corresponding QPC in the memory.

[0135] If no, the cache controller performs S809 to release the replaced QPC.

[0136] Through the above method, in addition to the way of expanding the capacity to ensure that the first storage area has sufficient storage space, the network card can also directly store the queue context associated with the service application of the specified service level in the second storage area other than the first storage area, and ensure that the storage duration of the queue context is consistent with the storage duration of the data in the first storage area, thereby ensuring that the queue context of the service of the specified service level can be stored in the network card for a long time, avoiding the problem of long processing delay caused by the frequent access of the network card to the memory of the device, and reducing the processing delay of the RDMA request to be processed by the service of the specified service level.

[0137] Next, the data processing method provided by the present application will be further explained in combination with a specific example, such as Figure 9As shown, service application 1 and service application 2 run simultaneously on the first processor 22 of the source device 20, wherein the service level of service application 1 is higher than that of service application 2. Taking the source device 20 reading data to the destination device as an example, the specific steps are as follows:

[0138] Step 1: Service program 1 generates RDMA read request 1 and stores it in SQ1, and service program 2 generates RDMA read request 2 and stores it in SQ2.

[0139] Step 2: The first processor notifies the network card that there are pending work requests in SQ1 and SQ2.

[0140] Step 3a: The second processor on the network card of the source device determines to execute the work request in SQ1 first according to the service level of the service application, and retrieves QPC1 of SQ1 in the memory. After the retrieval is unsuccessful, the second processor obtains QPC1 from the memory.

[0141] Step 4a: The second processor identifies the storage type of QPC1 as resident according to the service level of service application 1, and stores it in the resident cache.

[0142] Step 5a: The second processor obtains the RDMA read request 1 of service application 1 according to the QP address and QP address index in QPC1.

[0143] Step 6: The second processor sends the service level of service application 1 and the RDMA read request 1 to the communication interface.

[0144] Step 7: The communication interface sends the service level of service application 1 and the RDMA read request 1 to the network card 33 of the destination device 30.

[0145] Step 8: The network card 33 of the destination reads data from the memory according to the RDMA read request 1.

[0146] Step 9: The network card 33 of the destination returns the read data to the communication interface.

[0147] Step 10: The communication interface sends the read data to the second processor.

[0148] Step 11: The second processor retrieves CQC of the CQ in the memory. After the retrieval is successful, the second processor obtains CQC from the memory.

[0149] Step 12: The second processor puts the generated CQE into the CQ according to CQC.

[0150] After the above steps, the source completes the processing of the RDMA write request 1 of service application 1, and then the network card of the source continues to process the RDMA write request 2 of service application 2.

[0151] Step 3b: The network card of the source device retrieves QPC2 of SQ2 in the memory. After the retrieval is unsuccessful, the second processor obtains QPC2 from the memory.

[0152] Step 4b: The second processor identifies the storage type of QPC2 as non-resident according to the service level of the service application 2, and stores QPC2 in the non-resident cache.

[0153] Step 5b: The second processor obtains the RDMA read request 2 of the service application 2 according to the SQ address and the SQ address index in QPC2.

[0154] The network card then processes the RDMA read request 2 in a step similar to step 4-12.

[0155] Based on the above steps, the method for data processing provided by the present application can prioritize processing of work requests of service applications of high service levels, and can prioritize satisfying the cache space required by service applications of high service levels in the case of limited storage resources in the network card, thereby guaranteeing the transmission efficiency of important service applications.

[0156] Figure 10 is a flowchart of a method for modifying a QPC using application software provided by the present embodiment, and the specific method includes:

[0157] S1001: The application software process issues a command to modify the QP and notifies the intelligent network card through the doorbell module.

[0158] S1002: The execution unit on the intelligent network card retrieves whether the QPC is stored in the cache. If yes, the execution unit performs step S1003; otherwise, the execution unit performs step 1005 and notifies the application software that there is no information of the QPC in the cache and the QP can be modified.

[0159] S1003: The execution unit determines whether the cache of the current QPC is dirty data. If yes, the execution unit performs step 1004, writes the latest QPC in the current cache into the system memory, and notifies the application software that the QP can be modified; if not, the execution unit performs step S1006, releases the cache of the QPC, and notifies the application software that the QP can be modified.

[0160] Figure 11 is a flowchart of a method for deleting a QPC using application software provided by the present embodiment, and the specific method includes:

[0161] S1101: The application software process issues a command to delete the QP and notifies the network card through the doorbell module.

[0162] S1102, the execution unit on the network card retrieves whether the QPC is stored in the cache. If yes, the execution unit executes step S1103, releases the cache of the QPC, and reduces the number of cache entries corresponding to the priority by 1, while informing the application software that the QP can be deleted; otherwise, the execution unit executes step 114, informs the application software that there is no information of the QPC in the cache, and the QP can be deleted.

[0163] From the description of the above process, it can be known that the data processing method provided by the embodiments of the present application can provide a flexible interface for the application software, and facilitate the user to manage the network connection through the network management interface, including creating, deleting, querying and modifying the QP, and making relevant Qos configuration according to the actual communication demand.

[0164] It is worth noting that, for the above method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, and secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily required by the present application.

[0165] Those skilled in the art can think of other reasonable step combinations according to the above description, which also belong to the protection scope of the present application. Secondly, those skilled in the art should be familiar with the fact that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily required by the present application.

[0166] The above describes in detail the method of data transmission provided by the embodiments of the present application, and the following will describe a data transmission device and equipment provided by the embodiments of the present application in combination with Figures 6 to 11 Figures 12-13

[0167] Figure 12 A schematic diagram of a data transmission device 1200 provided by the present application includes an acquisition unit 1201, a processing unit 1202 and a communication unit 1203.

[0168] The acquisition unit 1201 is used for acquiring a first notification message and a second notification message, wherein the first notification message is used for indicating that there is a first to-be-processed remote direct memory access (RDMA) request in a first queue of a source device, the first queue is used for storing the RDMA request of a first service application in a remote device, and the second notification message is used for indicating that there is a second to-be-processed remote direct memory access (RDMA) request in a second queue of the source device, the second queue is used for storing the RDMA request of a second service application in the remote device.

[0169] ​​The processing unit 1202 determines the processing order of the first queue and the second queue according to the service level of the first service application and the service level of the second service application.

[0170] The communication unit 1203 is configured to send the first to-be-processed RDMA request and the second to-be-processed RDMA request to the destination device respectively according to the processing order.

[0171] It should be understood that the apparatus 1200 for data transmission in the embodiments of the present application can be implemented by a general processor (CPU), or an application-specific integrated circuit (ASIC), or a programmable logic device (PLD) such as a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The apparatus 1200 for data transmission can also be implemented by software. Figures 6 to 11 In the illustrated sensing method, the apparatus 1200 for data transmission and each module thereof can also be a software module.

[0172] Optionally, the processing unit 1202 is further configured to, when the processing order of the first service application is prior to the processing order of the second service application, acquire the first to-be-processed RDMA request, and send the first to-be-processed RDMA request to the destination device by using the communication unit 1203. Then, the processing unit continues to acquire the second to-be-processed request, and sends the second to-be-processed RDMA request to the destination device by using the communication unit 1203.

[0173] Optionally, the apparatus 1200 further includes a storage unit 1204. Before the network card sends the to-be-processed RDMA request in the first queue to the destination device, the processing unit 1202 is further configured to determine a storage manner of a context of the first queue in the storage according to a set rule. The context of the first queue is used to indicate an identifier of the destination device and an identifier of a queue associated with the first queue in the destination device. Then, the context of the first queue is acquired by the acquisition unit 1201, and the context of the first queue is stored in the storage unit 1204 of the network card by the processing unit 1202 according to the storage manner.

[0174] Optionally, the processing unit 1202 is further configured to determine the storage manner of the context of the first queue in the storage according to an operation type of the to-be-processed RDMA request in the first queue, wherein the operation type includes a persistent collective operation.

[0175] Optionally, the processing unit 1202 is further configured to compare a size of a data packet in the RDMA request to be processed in the first queue with a size of a first threshold, and determine a storage manner of the context of the first queue in the memory according to a comparison result.

[0176] Optionally, the processing unit 1202 is further configured to compare a usage frequency of the first queue with a second threshold, the first queue comprising a plurality of work queue elements (WQE), each WQE being configured to store one RDMA request to be processed, wherein the usage frequency is configured to identify a number of elements in the first queue storing the RDMA request to be processed within a first time threshold.

[0177] Optionally, the processing unit 1202 is further configured to determine the storage manner of the context of the first queue in the memory according to a service level of the first service application.

[0178] Optionally, the first queue and the context of the first queue are stored in a memory of the source device, and after the obtaining unit 1201 obtains the context of the first queue from the memory, the processing unit 1202 is further configured to store the context of the first queue in the storage unit 1204 according to the storage manner, including: the processing unit 1202 stores the context of the first queue to a first storage area of the storage unit 1204, and retains the context of the first queue in the first storage area within the first time threshold.

[0179] Optionally, the second queue and the context of the second queue are stored in the memory of the source device, and after the obtaining unit 1201 obtains the context of the second queue from the memory, the processing unit 1202 is further configured to store the context of the second queue in the storage unit 1204 according to the storage manner, including: the processing unit 1202 stores the context of the second queue to a second storage area of the storage unit 1204, and deletes the context of the second queue in the second storage area after completing the sending of the second RDMA request to be processed.

[0180] Optionally, the processing unit 1202 is further configured to obtain the context of the first queue or the context of the second queue from the memory of the source device through direct memory access (DMA).

[0181] Optionally, when a remaining space of the first storage area is less than a preset threshold, the processing unit 1202 is further configured to take a storage area of a third threshold size in the second storage area as an added storage area of the first storage area.

[0182] Optionally, when the remaining space of the first storage area is less than a preset threshold, the processing unit 1202 is further configured to store the context of the third queue to the second storage area of the memory of the network card, and the context of the third queue is kept in the second storage area within a first time threshold. The context of the third queue is used to indicate the identity of the destination device and the identity of a queue associated with the third queue in the destination device. The third queue is used to store the pending RDMA requests of the third service application.

[0183] Optionally, when the remaining space of the second storage area is less than a preset threshold, the processing unit 1202 is further configured to store the context of the second queue to the second storage area of the memory of the network card, and delete the context of the fourth queue in the second storage area. The context of the fourth queue is used to indicate the identity of the destination device and the identity of a queue associated with the fourth queue in the destination device. The fourth queue is used to store the pending RDMA requests of the fourth service application.

[0184] Optionally, when the remaining space of the second storage area is less than a preset threshold, the processing unit 1202 is further configured to store the context of the second queue to the second storage area of the memory of the network card, and delete the context of the fifth queue in the second storage area. The context of the fifth queue is used to indicate the identity of the destination device and the identity of a queue associated with the fifth queue in the destination device. The fifth queue is used to store the pending RDMA requests of the fifth service application, and the storage level of the context of the fifth queue is lower than that of the context of the second queue.

[0185] Optionally, when the remaining space of the second storage area is less than a preset threshold, the processing unit 1202 is further configured to store the context of the second queue to the second storage area of the memory of the network card, and delete the context of the sixth queue in the second storage area. The context of the sixth queue is used to indicate the identity of the destination device and the identity of a queue associated with the sixth queue in the destination device. The sixth queue is used to store the pending RDMA requests of the sixth service application, and the storage level of the context of the sixth queue is the lowest among the storage levels of the contexts of all the queues in the second storage area.

[0186] Optionally, the processing unit 1202 is further configured to determine whether the deleted context is dirty data. If yes, the processing unit 1202 stores the deleted context in the context cache area of the corresponding queue in the memory. If not, the processing unit 1202 directly deletes the deleted context. Through the above method, the network card can synchronize the queue context in the memory and the queue context in the memory in real time.

[0187] Optionally, the obtaining unit 1201 is further configured to obtain the first to-be-processed RDMA request from the memory of the source device, and the processing unit 1202 is configured to store the first to-be-processed RDMA request to the storage unit 1204. Then, the communication unit 1203 is further configured to send the first to-be-processed RDMA request to the destination device, and after sending the first to-be-processed request to the destination device, the obtaining unit 1202 obtains a second to-be-processed request, and the communication unit 1203 continues to send the second to-be-processed RDMA request to the destination device, including: the obtaining unit 1201 obtains the second to-be-processed RDMA request from the memory of the source device, the processing unit 1202 stores the second to-be-processed RDMA request to the storage unit 1204, and the communication unit 1203 sends the second to-be-processed RDMA request to the destination device.

[0188] The apparatus 1200 for data transmission according to the embodiments of the present application can correspond to performing the methods described in the embodiments of the present application, and the above and other operations and / or functions of each unit in the apparatus 1200 for data transmission are respectively for realizing the corresponding flows of each method in the embodiments of the present application, which will not be repeated here for brevity. Figures 6 to 11 The apparatus 1200 for data transmission according to the embodiments of the present application can correspond to performing the methods described in the embodiments of the present application, and the above and other operations and / or functions of each unit in the apparatus 1200 for data transmission are respectively for realizing the corresponding flows of each method in the embodiments of the present application, which will not be repeated here for brevity.

[0189] In summary, the apparatus 1200 for data transmission according to the embodiments of the present application can perceive the service level of the service application, determine the processing order of the queue according to the service level of the service application, and perform the data transmission of the to-be-processed RDMA request according to the processing order, so as to ensure that the service application of a specified service level can be preferentially sent to the destination device, thereby reducing the time delay in the data transmission of such service application.

[0190] Figure 13 A schematic diagram of a device 1300 according to an embodiment of the present application is shown in the figure. The device 1300 includes a first processor 1301, a first memory 1302, a network card 1303, a bus 1304 and a memory 1305. The first processor 1301, the first memory 1302, the network card 1303 and the memory 1305 communicate through the bus 1304, and can also communicate through wireless transmission or other means. The first processor 1301 is configured to run a plurality of service applications, each service application being associated with a queue, and the memory is configured to store remote direct memory access (RDMA) requests to be processed in the queue associated with each service application and the context of the queue associated with each service application.

[0191] The network card 1303 includes a second processor 1306, a second memory 1307 and a communication interface 1308. The second processor 1306 is configured to execute computer execution instructions stored in the second memory to perform the following operation steps:

[0192] acquire a first notification message, wherein the first notification message is used to indicate that there is a first to-be-processed remote direct memory access (RDMA) request in a first queue of the source device, and the first queue is used to store the RDMA request of a first service application in the remote device;

[0193] acquire a second notification message, wherein the second notification message is used to indicate that there is a second to-be-processed remote direct memory access (RDMA) request in a second queue of the source device, and the second queue is used to store the RDMA request of a second service application in the remote device;

[0194] determine a processing order of the first queue and the second queue according to a service level of the first service application and a service level of the second service application;

[0195] send the first to-be-processed RDMA request and the second to-be-processed RDMA request to the destination device respectively according to the processing order.

[0196] It should be understood that, in the embodiments of the present application, the first processor 1301 or the second processor 1306 can be a CPU, and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0197] The first memory 1302 or the second memory 1307 can include read-only memory and random access memory, and provide instructions and data for the processor 701. The first memory 1302 or the second memory 1307 can also include non-volatile random access memory. For example, the first memory 1302 or the second memory 1307 can also store device type information.

[0198] The first memory 1302 or the second memory 1307 can be volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically EPROM (EEPROM), or flash memory. The volatile memory can be random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate synchronous dynamic RAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0199] The bus 1304 can include, in addition to the data bus, a power bus, a control bus, and a state signal bus, etc. However, for the sake of clarity, all buses are shown as the bus 1304 in the figure.

[0200] It should be understood that the device 1300 according to the embodiments of the present application can correspond to the apparatus 1200 for data transmission in the embodiments of the present application, and can correspond to the source device performing the method in the embodiments of the present application, in addition, the device can also be any one of the devices in the system shown, and the above and other operations and / or functions of each module in the device 1300 are respectively for realizing the corresponding flow of each method in the figure, for the sake of brevity, it will not be repeated here. Figures 6 to 11 Figure 1 It should be understood that the device 1300 according to the embodiments of the present application can correspond to the apparatus 1200 for data transmission in the embodiments of the present application, and can correspond to the source device performing the method in the embodiments of the present application, in addition, the device can also be any one of the devices in the system shown, and the above and other operations and / or functions of each module in the device 1300 are respectively for realizing the corresponding flow of each method in the figure, for the sake of brevity, it will not be repeated here.

[0201] ​As described above, the device 1300 provided in this application embodiment can sense the service level of a service application, determine the processing order of multiple queues according to the service level of different service applications, and then complete the data transmission process of the pending RDMA requests of different queues according to the processing order. This realizes the process of the network card sensing the service level of the service application and sending the RDMA requests for processing of service applications of different service levels according to the service level, giving priority to ensuring that the pending RDMA requests of the specified service application can be quickly sent to the destination device, thereby reducing the processing latency of the service application of the specified service level.

[0202] This application also provides a chip including a processor and a communication interface, wherein the communication interface is used to communicate with the processor of the device in which the chip is located, and the processor can perform, for example... Figure 13 The network interface card 1302 shown is implemented with a second processor 1306. The processor of the chip is used to implement... Figures 6 to 11 For the sake of simplicity, the functions of the operation steps performed by the network interface card of the source device in the method shown will not be described in detail here.

[0203] Optionally, the chip can be designed to achieve the above. Figures 6 to 11 The chip that provides the network card function described above can also be... Figure 13 The network card 1303 shown can also be used for Figure 13 The device 1300 shown includes an offload card other than the network card 1303. This offload card is used to determine the processing order of different queues according to the service level of the business application, and then send the RDMA requests to be processed according to the determined processing order. For simplicity, it will not be described in detail here.

[0204] This application also provides a network interface card (NIC), the structure of which is as follows: Figure 13 The network card 1303 shown is used to implement... Figures 6 to 11 The functions of the operation steps performed by the network interface card of the source device in the method shown are not described in detail here for the sake of brevity.

[0205] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available medium sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0206] The above is only a specific implementation of the present application. Those skilled in the art can think of changes or replacements based on the specific implementation provided by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for data transmission, characterized in that, The method includes: The network interface card of the source device receives a first notification message, which indicates that there is a first pending Remote Direct Memory Access (RDMA) request in the first queue of the source device. The first queue is used to store requests from the first service application in the source device. The network card acquires a second notification message, which indicates that there is a second pending RDMA request in the second queue of the source device. The second queue is used to store requests from the second service application in the source device. The network interface card (NIC) determines the processing order of the first queue and the second queue based on the service level of the first service application and the service level of the second service application. The network interface card (NIC) determines the storage method of the context of the first queue in the memory according to the set rules. The context of the first queue is used to indicate the identifier of the destination device and the identifier of the queue associated with the first queue in the destination device. The storage method is used to indicate whether the context of the first queue is stored in the resident cache or the non-resident cache in the memory. The network interface card (NIC) obtains the context of the first queue; The network interface card (NIC) stores the context of the first queue in its memory according to the storage method described above. The network interface card sends the first pending RDMA request and the second pending RDMA request to the destination device in the processing order, respectively. The network interface card (NIC) determines the storage method of the context of the first queue in the memory according to the set rules, including at least one of the following methods: (1) The network card determines the storage method of the context of the first queue in the memory according to the operation type of the RDMA request to be processed in the first queue, wherein the operation type includes persistent collection operation; (2) The network card compares the size of the data packets in the pending RDMA requests in the first queue with the size of the first threshold; The network interface card determines the storage method of the context of the first queue in the memory based on the comparison result; (3) The network card compares the usage frequency of the first queue with the second threshold. The first queue includes multiple work queue elements (WQEs), each WQE is used to store a pending RDMA request. The usage frequency is used to identify the number of pending RDMA requests stored in each element of the first queue within a first time threshold. The network interface card (NIC) determines the storage method of the context of the first queue in the memory based on the comparison result.

2. The method according to claim 1, characterized in that, Sending the first pending RDMA request and the second pending RDMA request to the network card of the destination device according to the processing order includes: When the processing order of the first service application is before the processing order of the second service application, the network card obtains the first pending RDMA request and sends the first pending RDMA request to the destination device. After sending the first pending request to the destination device, the network card obtains the second pending request and sends the second pending request to the destination device.

3. The method according to claim 1, characterized in that, The network interface card (NIC) determines the storage method of the context of the first queue in the memory according to the set rules, including: The network interface card (NIC) determines the storage method of the context of the first queue in the memory based on the service level of the first service application.

4. The method according to claim 1, characterized in that, The first queue and its context are stored in the memory of the source device. The network interface card (NIC) obtains the context of the first queue, including: The network interface card (NIC) obtains the context of the first queue from the memory of the source device; The network interface card (NIC) then stores the context of the first queue in its memory according to the storage method, including: The network interface card (NIC) stores the context of the first queue in a first storage area of ​​the NIC's memory, and retains the context of the first queue in the first storage area within a first time threshold.

5. The method according to claim 2, characterized in that, The second queue and its context are stored in the memory of the source device. Before the network interface card (NIC) acquires the second pending RDMA request, the method further includes: The network interface card (NIC) obtains the context of the second queue from the memory of the source device; The network interface card (NIC) stores the context of the second queue in the second storage area of ​​the NIC's memory, and deletes the context of the second queue in the second storage area after the second pending RDMA request has been sent.

6. The method according to claim 4 or 5, characterized in that, The network interface card (NIC) obtains the context of the first queue from the memory of the source device, including: The network interface card (NIC) obtains the context of the first queue or the context of the second queue from the memory of the source device via Direct Memory Access (DMA).

7. The method according to claim 4 or 5, characterized in that, The method further includes: When the remaining space in the first storage area is less than the third threshold, the network card will use a storage area of ​​the fourth threshold size in the second storage area as a new storage area in the first storage area.

8. The method according to claim 4 or 5, characterized in that, The method further includes: When the remaining space in the first storage area is less than the third threshold, the network card stores the context of the third queue in the second storage area, and the context of the third queue is retained in the second storage area within the first time threshold. The context of the third queue is used to indicate the identifier of the destination device and the identifier of the queue associated with the third queue in the destination device. The third queue is used to store the pending RDMA requests of the third service application.

9. The method according to claim 2, characterized in that, The network interface card (NIC) acquires the first pending RDMA request and sends the first pending RDMA request to the destination device, including: The network interface card (NIC) obtains the first RDMA request to be processed from the memory of the source device and stores the first RDMA request to be processed in the memory of the NIC. The network interface card sends the first pending RDMA request to the destination device; After sending the first pending request to the destination device, the network card acquires the second pending request and sends the second pending request to the destination device, including: The network interface card (NIC) obtains the second RDMA request to be processed from the memory of the source device and stores the second RDMA request to be processed in the memory of the NIC. The network card sends the second pending RDMA request to the destination device.

10. The method according to claim 1, characterized in that, The network interface card (NIC) obtains the first pending RDMA request or the second pending RDMA request from the memory of the source device via Direct Memory Access (DMA).

11. The method according to any one of claims 1 to 5, 9 to 10, characterized in that, The network card is a smart network card that supports RDMA functionality.

12. The method according to any one of claims 1 to 5, 9 to 10, characterized in that, The network interface card (NIC) includes a doorbell register, which stores a first notification message from the first queue and a second notification message from the second queue. The first notification message includes the service level of the first service application, and the second notification message includes the service level of the second service application.

13. The method according to any one of claims 1 to 5, 9 to 10, characterized in that, The first queue includes any one of a send queue (SQ), a receive queue (RQ), or a completion queue (CQ), wherein the send queue and the receive queue form a queue pair, and the first queue context includes a queue pair context or a completion queue context.

14. A chip, characterized in that, The chip includes a processor and a communication interface. The communication interface is used to communicate with the processor of the device in which the chip is located. The processor of the chip is used to implement the operation steps of any of the methods described in claims 1 to 13.

15. A device for data transmission, characterized in that, The device for data transmission includes a first processor, memory, and a network interface card. The first processor is used to run multiple business applications, each business application is associated with a queue, and the memory is used to store remote direct memory access (RDMA) requests to be processed in the queues associated with the multiple business applications. The network card is used to implement the operational steps of any of the methods described in claims 1 to 13.

Images