RDMA-based single-QP scene performance optimization system and method

Abstract

The invention relates to the technical field of data transmission, in particular to a single-QP scene performance optimization system and method based on RDMA. The system comprises an HCA network card and a host which are electrically connected, wherein the HCA network card internally comprises a DBPROC module, a WQEPROC module and a PKTENG module which are electrically connected; a WQECACHE buffer memory space is arranged in the WQEPROC module; the DBPROC module obtains a scheduling request of a target QP; the WQEPROC module queries whether a target work queue element WQE corresponding to the target QP exists in a WQECACHE cache space or not; if yes, reading the data from the WQECACHE cache space; and if not, reading from the host. According to the invention, the delay of reading the WQE from the host is reduced, so that the bandwidth performance in a single QP scene is improved, and the data processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a single QP scenario performance optimization system and method based on RDMA. Background Art

[0002] RDMA (Remote Direct Memory Access) technology aims to eliminate many delay links in server-side data processing in traditional network transmission, realize direct access to remote memory data, and greatly improve data transmission efficiency.

[0003] In the RDMA-based network architecture, the HCA (Host Channel Adaptor) network card plays a key role. The HCA network card contains multiple functional modules that work together. The host usually sends task requests to DB_PROC (doorbell processing module), which is responsible for accurately dispatching task requests to WQE_PROC (WQE processing module). Based on the received task request, WQE_PROC reads WQE (Work Queue Element) from the host memory and performs deep processing, and then dispatches it to PKT_ENG (packet engine). PKT_ENG completes data reading and packetization of the header information carried by WQE based on the obtained descriptor and data address information, and finally realizes data transmission. However, the RDMA protocol and the HCA network card itself have specific working characteristics and limitations. On the one hand, the sending of RDMA task requests strictly follows the order preservation requirements, which means that the order of packet sending must be exactly the same as the order of the WQE sent. Therefore, within the same QP (Queue Pair), data processing can only be performed serially, and parallel processing of WQE cannot be achieved, which greatly limits the data processing speed in a single QP scenario. On the other hand, the HCA network card needs to support the execution of multiple QP tasks at the same time, and each QP shares the entire bandwidth resources. In order to avoid a QP being unable to obtain sufficient bandwidth resources for a long time, resulting in data transmission stagnation, each QP must perform a QP switching operation after sending a fixed amount of data.

[0004] In actual application scenarios, this feature brings obvious performance bottlenecks. In multi-QP scenarios, since multiple QPs can process tasks concurrently, the bandwidth resources of the HCA network card can be fully utilized, and the overall performance is friendly to the network card bandwidth. However, in single-QP scenarios, the bandwidth performance of the HCA network card is greatly limited, and its performance is completely dependent on the bandwidth performance of this one QP. In addition, since the number of QPs executed by the HCA network card during operation is in a dynamic state, it is difficult to accurately determine whether it is currently in a single-QP or multi-QP scenario, and thus it is impossible to implement a special optimization processing strategy for the single-QP scenario. Summary of the invention

[0005] In view of this, the present invention provides a single QP scenario performance optimization system and method based on RDMA to solve the problem of limited bandwidth performance in the existing solution in a single QP scenario.

[0006] In a first aspect, the present invention provides a single QP scenario performance optimization system based on RDMA, the system comprising an electrically connected HCA network card and a host, the HCA network card is connected to the host, the HCA network card internally comprises an electrically connected DB_PROC module, a WQE_PROC module and a PKT_ENG module; the WQE_PROC module is provided with a WQE_CACHE cache space;

[0007] The DB_PROC module is used to obtain the scheduling request of the target QP sent by the host;

[0008] The WQE_PROC module is used to query whether there is a target work queue element WQE corresponding to the target QP in the WQE_CACHE cache space based on the scheduling request of the target QP; if the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, read the target work queue element WQE corresponding to the target QP from the WQE_CACHE cache space; if the target work queue element WQE corresponding to the target QP does not exist in the WQE_CACHE cache space, read the target work queue element WQE corresponding to the target QP from the memory of the host; and process the read target work queue elements WQE one by one;

[0009] The PKT_ENG module is used to generate a data packet based on the target work queue element WQE after processing, and send the data packet to the target device.

[0010] In a second aspect, the present invention provides a single QP scenario performance optimization method based on RDMA, the method is applied to the HCA network card as described above, and the method includes:

[0011] Obtaining a scheduling request for a target QP issued by the host;

[0012] Based on the scheduling request of the target QP, query whether there is a target work queue element WQE corresponding to the target QP in the WQE_CACHE cache space;

[0013] If the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, then reading the target work queue element WQE corresponding to the target QP from the WQE_CACHE cache space;

[0014] If the target work queue element WQE corresponding to the target QP does not exist in the WQE_CACHE cache space, the target work queue element WQE corresponding to the target QP is read from the memory of the host.

[0015] In an optional implementation, before obtaining the scheduling request of the target QP sent by the host, the method further includes:

[0016] When executing the processing task of the target QP for the first time, parsing the context information of the scheduling request of the target QP to obtain the WQE read address;

[0017] Based on the WQE read address, read the initial work queue element WQE corresponding to the target QP from the host;

[0018] Process the read initial work queue elements WQE one by one, and determine whether the amount of data sent by the target QP reaches a target threshold;

[0019] If the amount of data sent by the target QP reaches the target threshold, determining whether there are any remaining unprocessed WQEs in the read initial work queue element WQE;

[0020] If there are remaining WQEs that have not been processed yet, and the WQE_CACHE buffer space is not occupied, the remaining WQEs that have not been processed yet are stored in the WQE_CACHE buffer space, waiting for the next scheduling request of the target QP.

[0021] In an optional implementation, after the target work queue element WQE corresponding to the target QP is read, the method further includes:

[0022] Process the read target work queue elements WQE one by one, and determine whether the amount of data sent by the target QP reaches the target threshold;

[0023] If the amount of data sent by the target QP reaches the target threshold, determining whether the WQE_CACHE cache space is free;

[0024] If the WQE_CACHE cache space is free, the remaining unprocessed WQEs in the read target work queue element WQE are stored again in the WQE_CACHE cache space to wait for the next scheduling request of the target QP.

[0025] In an optional embodiment, the method further includes:

[0026] If the amount of data sent by the target QP does not reach the target threshold, the read target work queue elements WQE continue to be processed one by one.

[0027] In an optional embodiment, the method further includes:

[0028] If the WQE_CACHE cache space is not free, the remaining unprocessed WQEs in the read target work queue element WQE are drained.

[0029] In an optional embodiment, the method further includes:

[0030] Dispatching the processed target work queue element WQE to the PKT_ENG module, so that the PKT_ENG module generates a data packet based on the processed target work queue element WQE, and sends the data packet to the target device;

[0031] Activates the specified QP in the DB_PROC module to wait for a rescheduling request.

[0032] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to implement a single QP scenario performance optimization method based on RDMA according to the second aspect or any corresponding embodiment thereof.

[0033] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed by a processor, a single QP scenario performance optimization method based on RDMA is implemented according to the second aspect or any corresponding embodiment thereof.

[0034] In a fifth aspect, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute a single QP scenario performance optimization method based on RDMA according to the above-mentioned second aspect or any corresponding embodiment thereof.

[0035] The technical solution provided by the present invention may include the following beneficial effects:

[0036] The present invention integrates the DB_PROC module, the WQE_PROC module and the PKT_ENG module in the HCA network card, and sets the WQE_CACHE cache space, so that the system can respond to the scheduling request of the target QP issued by the host more quickly. When the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, the system can read directly from the cache, avoiding the delay of reading from the host memory, thereby improving the processing efficiency. The present invention reduces the delay of reading WQE from the host by introducing the WQE_CACHE cache mechanism inside the WQE_PROC module, thereby improving the bandwidth performance in the single QP scenario, and the efficiency of data processing is improved, which is beneficial to the application scenario requiring low latency and high throughput. This optimization scheme is not only applicable to single QP scenarios, but also can adapt to multi-QP scenarios. There is no need to distinguish whether it is a single QP scenario. The WQE_CACHE cache mechanism can automatically adapt and optimize performance. By reducing the WQE read delay in the core mechanism of data transmission and operation, the overall network transmission delay is reduced. Moreover, in a single QP scenario, the larger the WQE message, the fewer WQEs consumed in one scheduling, the more WQEs cached in WQE_CACHE, the more QP scheduling times can be supported, and the better the performance optimization effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a schematic diagram of the internal structure of an HCA network card according to an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of the internal processing flow of the WQE_PROC module according to an embodiment of the present invention;

[0040] Figure 3 It is a structural diagram of a single QP scenario performance optimization system based on RDMA according to an embodiment of the present invention;

[0041] Figure 4 It is a flowchart of a single QP scenario performance optimization method based on RDMA according to an embodiment of the present invention;

[0042] Figure 5 is a flow chart of another RDMA-based single QP scenario performance optimization method according to an embodiment of the present invention;

[0043] Figure 6 is a flowchart of another RDMA-based single QP scenario performance optimization method according to an embodiment of the present invention;

[0044] Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] It should be noted that some key terms in the present invention are first explained:

[0047] HCA (Host Channel Adaptor): HCA (Host Channel Adaptor) is a network card that plays a key role in the connection of computer network systems. Its main function is to establish a high-speed data transmission channel between the host system and the network, and is responsible for processing data exchange between the host and external network devices (such as other servers, storage devices, etc.).

[0048] RDMA (Remote Direct Memory Access): RDMA (Remote Direct Memory Access) is a technology that is designed to solve the delay in server-side data processing during network transmission. In a distributed storage system, the application of RDMA technology can make data transmission between storage nodes faster, improving the performance and response speed of the entire system.

[0049] QP (Queue Pair): In the InfiniBand communication architecture, QP is the core logical entity. It consists of a send queue and a receive queue, which work together to manage the data sending and receiving process. The send queue is responsible for storing the data to be sent and its related operation instructions, and sending the data in a certain order; the receive queue is used to receive the incoming data and process it according to pre-set rules.

[0050] WQE: WQE (Work Queue Element) is a data structure used to define operations in the InfiniBand work queue, such as data sending, receiving, atomic operations, etc. It is the core mechanism for InfiniBand to implement data transmission and operations.

[0051] DB_PROC module (doorbell processing module): The DB_PROC module is mainly responsible for processing task requests issued by the host, receiving various operation instructions from the host, and assigning tasks to corresponding modules for processing according to these instructions. When the host has a data transmission requirement, it will send a request to the DB_PROC module, and the DB_PROC module will dispatch it to the WQE_PROC module to trigger the subsequent data processing process.

[0052] WQE_PROC module (WQE processing module): The WQE_PROC module is a key link in the entire data processing process. According to the task request received from the DB_PROC module, it reads the WQE from the host memory, parses and processes it. After the processing is completed, the processing results or data that needs to be further sent are dispatched to the PKT_ENG module for data packet encapsulation and transmission.

[0053] PKT_ENG module (packet sending engine): The main responsibility of the PKT_ENG module is to assemble the data and the packet header information carried by WQE according to the received descriptor and data address information and send them to the network.

[0054] WQE_CACHE cache space (on-chip WQE cache space): WQE_CACHE cache space is a cache area located on the chip, which is used to store excess WQEs read from the host. When the system is processing the data of a certain QP, if there are surplus WQEs read from the host and the WQE_CACHE cache space is not occupied, these excess WQEs will be stored in the WQE_CACHE cache space. In this way, when the QP is scheduled to read the WQE operation next time, if there is a corresponding WQE in the WQE_CACHE cache space, it can be read directly from the cache, which greatly reduces the delay of reading the WQE from the host, thereby improving the overall performance of the system.

[0055] Bandwidth: In digital devices, bandwidth refers to the amount of data that can pass through a link per unit time. It is usually expressed in bps, which is the number of bits that can be transmitted per second.

[0056] See also Figure 1The internal structure diagram of the HCA network card is shown in Figure 1. In the relevant scheme, the host CPU sends a task request to the DB_PROC module, and the DB_PROC module dispatches the task request to the WQE_PROC module. The WQE_PROC module reads the WQE from the host memory according to the received task request, processes it, and dispatches it to the PKT_ENG module. The PKT_ENG module reads the data and the packet header information carried by the WQE according to the received descriptor and data address information, and then sends it to the target device. See Figure 2 The internal processing flow diagram of the WQE_PROC module shown in the figure shows that after receiving the QP request, the context is parsed to obtain the WQE read address, a fixed number of WQEs are read and cached locally, the WQEs are processed one by one, and the data volume is judged to determine whether the amount of data sent reaches 64KB. If not, return to the step of reading WQEs one by one for processing and continue to process the next WQE. If yes, proceed to the next step. After that, the excess WQEs are drained, and the packets are scheduled to the PKT_ENG module, the QP in the DB_PROC module is activated, and waits for rescheduling. However, due to the specific working characteristics of the RDMA protocol and the HCA network card itself, there are restrictions. In the multi-QP scenario, QPs are processed concurrently, which is friendly to the HCA network card bandwidth; in the single QP scenario, the bandwidth performance of the HCA network card is limited and completely depends on the bandwidth performance of this QP. And the number of QPs currently being executed by the HCA network card is dynamically changing and cannot be accurately judged, so it is impossible to do separate processing for the single QP scenario.

[0057] To support flexible NIC cascading, HCA NICs need to meet the 400Gbps specification requirement in the WQE read latency scenario (2000ns). In a single QP scenario, the QP processing time is: internal processing latency + WQE read waiting time = 2100ns (internal processing latency is calculated as 100ns), and the bandwidth is: 64*1024*8 / 2100ns = 250G. It can be seen that the relevant solution cannot meet the bandwidth requirements.

[0058] After analysis, it can be known that the biggest factor limiting bandwidth in a single QP scenario is the delay in reading WQE from the host, so the present invention can improve bandwidth by shortening this delay. The present invention reduces the delay in reading WQE from the host by introducing the WQE_CACHE cache mechanism inside the WQE_PROC module, thereby improving the bandwidth performance in a single QP scenario, and the efficiency of data processing is improved, which is beneficial to application scenarios that require low latency and high throughput. This optimization scheme is not only applicable to single QP scenarios, but also to multi-QP scenarios. There is no need to distinguish whether it is a single QP scenario. The WQE_CACHE cache mechanism can automatically adapt and optimize performance. By reducing the WQE reading delay in the core mechanism of data transmission and operation, the overall network transmission delay is reduced. Moreover, in a single QP scenario, the larger the WQE message, the fewer WQEs consumed in one scheduling, the more cached in WQE_CACHE, the more QP scheduling times that can be supported, and the better the performance optimization effect.

[0059] Figure 3 is a schematic diagram of the structure of a single QP scenario performance optimization system based on RDMA according to an embodiment of the present invention, such as Figure 3 As shown, the system includes an electrically connected HCA network card and a host, the HCA network card is connected to the host, and the HCA network card includes an electrically connected DB_PROC module, a WQE_PROC module and a PKT_ENG module; the WQE_PROC module is provided with a WQE_CACHE cache space;

[0060] The DB_PROC module is used to obtain the scheduling request of the target QP sent by the host;

[0061] The WQE_PROC module is used to query whether there is a target work queue element WQE corresponding to the target QP in the WQE_CACHE cache space based on the scheduling request of the target QP; if the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, read the target work queue element WQE corresponding to the target QP from the WQE_CACHE cache space; if the target work queue element WQE corresponding to the target QP does not exist in the WQE_CACHE cache space, read the target work queue element WQE corresponding to the target QP from the memory of the host; and process the read target work queue elements WQE one by one;

[0062] The PKT_ENG module is used to generate a data packet based on the processed target work queue element WQE, and send the data packet to the target device.

[0063] Furthermore, in this embodiment, the WQE_CACHE cache space is used inside the WQE_PROC module to cache the WQEs that need to be fetched more. The next time the QP is scheduled, it is directly read from the local to reduce the reading delay. When the WQE_CACHE cache space is not occupied, the fetched WQEs are cached in the WQE_CACHE cache space. This caching mechanism does not need to distinguish whether it is a single QP scenario. The WQE_CACHE cache space can reduce the WQE reading delay and improve the bandwidth performance.

[0064] In summary, by integrating the DB_PROC module, the WQE_PROC module, and the PKT_ENG module in the HCA network card and setting the WQE_CACHE cache space, the system can respond to the scheduling request of the target QP issued by the host more quickly. When the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, the system can read directly from the cache, avoiding the delay of reading from the host memory, thereby improving the processing efficiency. By introducing the WQE_CACHE cache mechanism inside the WQE_PROC module, the delay of reading WQE from the host is reduced, thereby improving the bandwidth performance in the single QP scenario, the efficiency of data processing is improved, and it is beneficial to the application scenario requiring low latency and high throughput. This optimization scheme is not only applicable to single QP scenarios, but also to multi-QP scenarios. There is no need to distinguish whether it is a single QP scenario. The WQE_CACHE cache mechanism can automatically adapt and optimize performance. By reducing the WQE read delay in the core mechanism of data transmission and operation, the overall network transmission delay is reduced. Moreover, in a single QP scenario, the larger the WQE message, the fewer WQEs are consumed in one scheduling, the more WQEs are cached in WQE_CACHE, the more QP scheduling times can be supported, and the better the performance optimization effect.

[0065] According to an embodiment of the present invention, an embodiment of a single QP scenario performance optimization method based on RDMA is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0066] In this embodiment, a single QP scenario performance optimization method based on RDMA is provided. Figure 3 In the HCA network card in the single QP scenario performance optimization system based on RDMA shown, the HCA network card is connected to the host, and the HCA network card includes an electrically connected DB_PROC module, a WQE_PROC module and a PKT_ENG module; the WQE_PROC module is provided with a WQE_CACHE cache space, Figure 4 FIG. 4 is a flow chart of a single QP scenario performance optimization method based on RDMA according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0067] Step S401: Obtain a scheduling request of a target QP sent by the host.

[0068] Further, step S401 is the starting step of the entire performance optimization method. When the host needs to perform data transmission or processing tasks, it sends a scheduling request for the target QP (queue pair) to the DB_PROC module in the HCA network card. The scheduling request is a trigger signal, indicating that the host wants to perform corresponding data operations, such as sending or receiving data, through the target QP. This step S401 is not the first execution of the processing task of the target QP, and can be any execution of the processing task of the target QP except the first time, such as the second time, the third time, the Nth time, etc.

[0069] Step S402: Based on the scheduling request of the target QP, query whether there is a target work queue element WQE corresponding to the target QP in the WQE_CACHE cache space.

[0070] Furthermore, after receiving the scheduling request, the system of this embodiment will search the WQE_CACHE cache space in the WQE_PROC module according to the information of the target QP to see whether the target work queue element WQE related to the target QP has been stored. WQE is a work queue element, which contains a specific description of data operations, such as information about sending and receiving operations. In the optimization process, the cache space is searched first to reduce the delay of reading from the host memory and improve performance. Since reading data from the cache is usually much faster than reading from the host memory, this embodiment first tries to search in the WQE_CACHE cache space, hoping to quickly find the required WQE, thereby improving overall performance.

[0071] Step S403: If the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, the target work queue element WQE corresponding to the target QP is read from the WQE_CACHE cache space.

[0072] Furthermore, if it is found in step S402 that the WQE corresponding to the target QP is indeed stored in the WQE_CACHE cache space, the WQE is directly read from the WQE_CACHE cache space to avoid the overhead of accessing the host memory and speed up the data processing. Since the speed of reading data from the WQE_CACHE cache space is relatively faster, the subsequent data operation process can be entered faster, so this embodiment reduces the time required for reading data from the host memory and improves the efficiency of data processing.

[0073] Step S404: if the target work queue element WQE corresponding to the target QP does not exist in the WQE_CACHE cache space, the target work queue element WQE corresponding to the target QP is read from the memory of the host.

[0074] Furthermore, when the WQE corresponding to the target QP cannot be found in the WQE_CACHE cache space, it means that there is no cache or the cache has expired. At this time, the required WQE needs to be obtained from the host's memory. The host memory stores the original data and operation information required by the system. Although the reading speed is relatively slow, it can ensure that the required data is obtained to ensure that subsequent data operations can proceed normally.

[0075] In summary, by integrating the DB_PROC module, the WQE_PROC module, and the PKT_ENG module in the HCA network card and setting the WQE_CACHE cache space, the system can respond to the scheduling request of the target QP issued by the host more quickly. When the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, the system can read directly from the cache, avoiding the delay of reading from the host memory, thereby improving the processing efficiency. By introducing the WQE_CACHE cache mechanism inside the WQE_PROC module, the delay of reading WQE from the host is reduced, thereby improving the bandwidth performance in the single QP scenario, the efficiency of data processing is improved, and it is beneficial to the application scenario requiring low latency and high throughput. This optimization scheme is not only applicable to single QP scenarios, but also to multi-QP scenarios. There is no need to distinguish whether it is a single QP scenario. The WQE_CACHE cache mechanism can automatically adapt and optimize performance. By reducing the WQE read delay in the core mechanism of data transmission and operation, the overall network transmission delay is reduced. Moreover, in a single QP scenario, the larger the WQE message, the fewer WQEs are consumed in one scheduling, the more WQEs are cached in WQE_CACHE, the more QP scheduling times can be supported, and the better the performance optimization effect.

[0076] In this embodiment, another single QP scenario performance optimization method based on RDMA is provided. Figure 3In the HCA network card in the single QP scenario performance optimization system based on RDMA shown, the HCA network card is connected to the host, and the HCA network card includes an electrically connected DB_PROC module, a WQE_PROC module and a PKT_ENG module; the WQE_PROC module is provided with a WQE_CACHE cache space, Figure 5 FIG. 4 is a flow chart of another RDMA-based single QP scenario performance optimization method according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0077] Step S501, when executing the processing task of the target QP for the first time, parse the context information of the scheduling request of the target QP to obtain the WQE reading address; process the read initial work queue elements WQE one by one, and determine whether the amount of data sent by the target QP reaches the target threshold.

[0078] Furthermore, when the task of the target QP is processed for the first time, the system will read the required initial work queue element WQE from the host. At this time, it is necessary to parse the context information of the scheduling request of the target QP for the first time to obtain the WQE read address, and then read the required initial work queue element WQE from the host according to the WQE read address. After that, the read initial work queue elements WQE are processed one by one, and it is determined whether the amount of data sent by the target QP reaches the target threshold; the target threshold can be 64KB, that is, the amount of data sent by the target QP reaches 64KB.

[0079] Step S502: If the amount of data sent by the target QP reaches the target threshold, it is determined whether there are any unprocessed remaining WQEs in the read initial work queue element WQE; if there are any unprocessed remaining WQEs and the WQE_CACHE cache space is not occupied, the unprocessed remaining WQEs are stored in the WQE_CACHE cache space to wait for the next scheduling request of the target QP.

[0080] Furthermore, this step is intended to check whether there are any unprocessed parts of the WQE read from the host. Since the system may read multiple WQEs for subsequent operations according to certain policies or rules, there may be some remaining after processing some WQEs. If it is found that there are unprocessed remaining WQEs and there is free space in the WQE_CACHE cache space, these unprocessed remaining WQEs are stored in the cache. This is done to improve performance, because subsequent processing of the same target QP may require these WQEs, and obtaining them from the cache is faster than reading from the host memory.

[0081] Step S503: Obtain the scheduling request of the target QP sent by the host. Figure 4 Step S401 of the illustrated embodiment will not be described in detail here.

[0082] Step S504: Based on the scheduling request of the target QP, query whether there is a target work queue element WQE corresponding to the target QP in the WQE_CACHE cache space.

[0083] In an optional implementation, the step S504 includes:

[0084] Parse the context information of the scheduling request of the target QP to obtain the WQE read address;

[0085] Based on the WQE read address, it is queried whether there is a target work queue element WQE corresponding to the target QP in the WQE_CACHE cache space.

[0086] For further information, see Figure 6 The flowchart of another single QP scenario performance optimization method based on RDMA is shown. After receiving the subsequent scheduling request of the target QP, this embodiment checks whether the corresponding WQE is stored in the WQE_CACHE cache space according to the information of the target QP. This step is the key to performance optimization, which determines whether the cache can be used to quickly obtain the WQE. This embodiment obtains the WQE read address by parsing the context information of the scheduling request, and then searches the corresponding WQE in the cache based on the address, thereby improving the accuracy and efficiency of the search.

[0087] Step S505: If the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, the target work queue element WQE corresponding to the target QP is read from the WQE_CACHE cache space.

[0088] Furthermore, if the WQE corresponding to the target QP is found in the WQE_CACHE cache space, it is directly read from the cache, avoiding the delay of reading from the host memory and improving the operation speed. In this embodiment, reading from the cache can quickly obtain the required WQE, speeding up the entire data processing flow, especially in high concurrency or delay-sensitive scenarios, which can significantly improve performance.

[0089] Step S506: If the target work queue element WQE corresponding to the target QP does not exist in the WQE_CACHE cache space, the target work queue element WQE corresponding to the target QP is read from the memory of the host.

[0090] Furthermore, when the WQE_CACHE cache space does not hit, this embodiment obtains the required WQE from the host memory. Although reading from the host memory is relatively slow, it can ensure that the correct data information is obtained to ensure the normal progress of subsequent operations.

[0091] Step S507, processing the read target work queue elements WQE one by one, and determining whether the amount of data sent by the target QP reaches the target threshold; if the amount of data sent by the target QP reaches the target threshold, determining whether there is any free cache space in the WQE_CACHE.

[0092] In an optional implementation, if the amount of data sent by the target QP does not reach the target threshold, the read target work queue elements WQE continue to be processed one by one.

[0093] Furthermore, the present embodiment first processes the read WQE, which may include data sending, receiving or other operations. At the same time, the amount of data sent by the target QP is continuously counted during the processing and compared with the set target threshold. When the target threshold is reached, it is necessary to consider whether there is space to store the excess WQE in the cache. The target threshold is set to determine when to perform the cache operation to optimize system performance. Reaching the target threshold means that a certain amount of data has been processed, and at this time, it can be considered whether to cache the remaining WQE. The target threshold can be 64KB, that is, the amount of data sent by the target QP reaches 64KB.

[0094] Step S508: If the WQE_CACHE cache space is free, the remaining unprocessed WQEs in the read target work queue element WQE are stored in the WQE_CACHE cache space again, waiting for the next scheduling request of the target QP. If the WQE_CACHE cache space is not free, the remaining unprocessed WQEs in the read target work queue element WQE are drained.

[0095] Further, when the target threshold is reached and WQE_CACHE has free space, this embodiment stores the unprocessed WQEs in the cache for subsequent use. If there is no free space, these unprocessed WQEs are drained to avoid wasting resources or affecting subsequent operations. This embodiment performs different processing according to the situation of the cache space, ensuring the effective use of the cache space and the optimization of system performance, while avoiding performance degradation caused by insufficient cache space.

[0096] In an optional implementation, the processed target work queue element WQE is scheduled to the PKT_ENG module so that the PKT_ENG module generates a data packet based on the processed target work queue element WQE and sends the data packet to the target device; the specified QP in the DB_PROC module is activated to wait for a rescheduling request.

[0097] Furthermore, after completing the processing of WQE, this embodiment will hand over the processing result to the PKT_ENG module, and the PKT_ENG module will generate and send out a data packet that complies with the network protocol based on the processed WQE information. At the same time, the specified QP in the DB_PROC module is activated to wait for the next scheduling to achieve continuous data processing and transmission operations. This step is an important step to connect the data processing and network transmission links, ensure that the data can be sent out correctly, and prepare for the next task.

[0098] The following is an explanation of the contents disclosed in the above embodiment through simple examples:

[0099] This example analyzes the performance of a work queue element WQE with a message size of 4 KB, and compares the performance difference before and after using the WQE_CACHE cache space in a single QP (queue pair) scenario.

[0100] 1. Analysis before optimization:

[0101] Read latency and scheduling duration: 32 WQEs (each WQE message is 4KB) are read from the host, and the read latency is 2000ns. The duration of one scheduling is composed of the internal processing latency (100ns) and the latency of reading WQEs from the host (2000ns), that is, scheduling duration = 100ns + 2000ns = 2100ns.

[0102] Bandwidth calculation: The amount of data processed in one scheduling process is 64KB (because 64 / 4=16 WQEs are processed, and each WQE is 4K). According to the bandwidth calculation formula (bandwidth=data volume / time), the WQE bandwidth before optimization can be obtained as 64KB / 2100ns≈249Gbps.

[0103] 2. Analysis after optimization:

[0104] Read latency and scheduling duration: First scheduling: 32 WQEs are read from the host, with a read latency of 2000ns. After 16 WQEs are processed, the remaining 16 WQEs are written to the WQE_CACHE cache space.

[0105] Second scheduling: Switch the WQE source to the WQE_CACHE cache space, and the WQE read delay is 2clk (negligible). At this time, the scheduling duration is mainly determined by the internal processing delay, that is, the duration of two QP scheduling = 2*internal processing delay (100ns) + 2000ns = 2200ns.

[0106] Bandwidth calculation: The total amount of data processed in the two schedulings is 128KB (64KB in the first processing and 64KB in the second processing), so the optimized bandwidth = 128KB / 2200ns≈476Gbps.

[0107] By comparison, it can be seen that after optimization, the bandwidth has increased from 249Gbps ​​to 476Gbps, and the performance has been significantly improved. The main reason is that the WQE_CACHE cache space is utilized to reduce the delay in reading WQE from the host, especially in the second scheduling, the delay in reading WQE from the cache is almost negligible, thereby greatly improving the overall bandwidth performance. In a single QP scenario, the larger the WQE message, the fewer WQEs consumed in one scheduling, so under the same reading volume, the more WQEs that can be stored in the WQE_CACHE cache space, which can support more QP scheduling to read WQE from the cache, thereby further improving the performance optimization effect.

[0108] In summary, by integrating the DB_PROC module, the WQE_PROC module, and the PKT_ENG module in the HCA network card and setting the WQE_CACHE cache space, the system can respond to the scheduling request of the target QP issued by the host more quickly. When the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, the system can read directly from the cache, avoiding the delay of reading from the host memory, thereby improving the processing efficiency. By introducing the WQE_CACHE cache mechanism inside the WQE_PROC module, the delay of reading WQE from the host is reduced, thereby improving the bandwidth performance in the single QP scenario, the efficiency of data processing is improved, and it is beneficial to the application scenario requiring low latency and high throughput. This optimization scheme is not only applicable to single QP scenarios, but also to multi-QP scenarios. There is no need to distinguish whether it is a single QP scenario. The WQE_CACHE cache mechanism can automatically adapt and optimize performance. By reducing the WQE read delay in the core mechanism of data transmission and operation, the overall network transmission delay is reduced. Moreover, in a single QP scenario, the larger the WQE message, the fewer WQEs are consumed in one scheduling, the more WQEs are cached in WQE_CACHE, the more QP scheduling times can be supported, and the better the performance optimization effect.

[0109] The embodiment of the present invention also provides a computer device, see Figure 7 , Figure 7 1 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, and the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0110] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0111] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0112] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0113] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0114] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0115] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0116] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0117] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the defined scope.

Claims

1. A single QP scenario performance optimization system based on RDMA, characterized in that: The system includes an electrically connected HCA network card and a host, wherein the HCA network card includes an electrically connected DB_PROC module, a WQE_PROC module and a PKT_ENG module; a WQE_CACHE cache space is provided in the WQE_PROC module; The DB_PROC module is used to obtain the scheduling request of the target QP sent by the host; The WQE_PROC module is used to query whether there is a target work queue element WQE corresponding to the target QP in the WQE_CACHE cache space based on the scheduling request of the target QP; If the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, then reading the target work queue element WQE corresponding to the target QP from the WQE_CACHE cache space; If the target work queue element WQE corresponding to the target QP does not exist in the WQE_CACHE cache space, the target work queue element WQE corresponding to the target QP is read from the memory of the host; and the read target work queue elements WQE are processed one by one; The PKT_ENG module is used to generate a data packet based on the target work queue element WQE after processing, and send the data packet to the target device.

2. A single QP scenario performance optimization method based on RDMA, characterized in that: The method is applied to the HCA network card according to claim 1, and the method comprises: Get the scheduling request of the target QP sent by the host; Based on the scheduling request of the target QP, query whether there is a target work queue element WQE corresponding to the target QP in the WQE_CACHE cache space; If the target work queue element WQE corresponding to the target QP exists in the WQE_CACHE cache space, then reading the target work queue element WQE corresponding to the target QP from the WQE_CACHE cache space; If the target work queue element WQE corresponding to the target QP does not exist in the WQE_CACHE cache space, the target work queue element WQE corresponding to the target QP is read from the memory of the host.

3. The method according to claim 2, characterized in that Before obtaining the scheduling request of the target QP sent by the host, the method further includes: When executing the processing task of the target QP for the first time, parsing the context information of the scheduling request of the target QP to obtain the WQE read address; Based on the WQE read address, read the initial work queue element WQE corresponding to the target QP from the host; Process the read initial work queue elements WQE one by one, and determine whether the amount of data sent by the target QP reaches a target threshold; If the amount of data sent by the target QP reaches the target threshold, determining whether there are any remaining unprocessed WQEs in the read initial work queue element WQE; If there are remaining WQEs that have not been processed yet, and the WQE_CACHE buffer space is not occupied, the remaining WQEs that have not been processed yet are stored in the WQE_CACHE buffer space, waiting for the next scheduling request of the target QP.

4. The method according to claim 3, characterized in that After the target work queue element WQE corresponding to the target QP is read, the method further includes: Process the read target work queue elements WQE one by one, and determine whether the amount of data sent by the target QP reaches the target threshold; If the amount of data sent by the target QP reaches the target threshold, determining whether the WQE_CACHE cache space is free; If the WQE_CACHE cache space is free, the remaining unprocessed WQEs in the read target work queue element WQE are stored again in the WQE_CACHE cache space to wait for the next scheduling request of the target QP.

5. The method according to claim 4, characterized in that The method further comprises: If the amount of data sent by the target QP does not reach the target threshold, the read target work queue elements WQE continue to be processed one by one.

6. The method according to claim 4, characterized in that The method further comprises: If the WQE_CACHE cache space is not free, the remaining unprocessed WQEs in the read target work queue element WQE are drained.

7. The method according to claim 4, characterized in that The method further comprises: Dispatching the processed target work queue element WQE to the PKT_ENG module, so that the PKT_ENG module generates a data packet based on the processed target work queue element WQE, and sends the data packet to the target device; Activates the specified QP in the DB_PROC module to wait for a rescheduling request.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements a single QP scenario performance optimization method based on RDMA as described in any one of claims 2 to 7 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by the processor, the RDMA-based single QP scenario performance optimization method according to any one of claims 2 to 7 is implemented.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute a single QP scenario performance optimization method based on RDMA as described in any one of claims 2 to 7.

Images