A WQE adaptive scheduling system and method

Abstract

The present invention relates to the technical field of data transmission, and in particular to a WQE adaptive scheduling system and method, the system comprising: a host, used to generate doorbell information, and send it to a DB module through a PCIE module; a PCIE module, used to establish a data transmission channel between the host and other modules; a DB module, used to receive and save the doorbell information, and calculate the number of QP doorbells in the chain based on the doorbell information, and output the number of QP doorbells in the chain to a WQE_PROC module; a WQE_PROC module, used to calculate the actual number of QP doorbells based on the number of QP doorbells in the chain, and dynamically adjust the number of WQE acquisitions requested from the host according to the actual number of QP doorbells. The present invention dynamically adjusts the number of WQE acquisitions each time based on the actual number of QP doorbells, improves the data transmission efficiency, and meets the application scenario requirements with high performance requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a WQE adaptive scheduling system and method. Background Art

[0002] With the rapid development of computer network technology, data transmission speed and efficiency have become important indicators for measuring network communication performance. Especially in the field of high-performance computing (HPC) and data centers, the requirements for data transmission bandwidth and low latency are increasing. As a high-performance and scalable network communication technology, InfiniBand has been widely used in high-performance computing and large-scale data centers due to its high bandwidth, low latency and efficient queue pair (QP) management mechanism.

[0003] However, in practical applications, especially in the scenario with few QPs, the performance optimization of InfiniBand faces some challenges. QP is the basic unit in InfiniBand communication, which is used to send and receive data. In the scenario with few QPs, that is, when the number of QPs in the system is small, if the traditional WQE (Work Queue Element) scheduling strategy is still used for data transmission, it may lead to insufficient utilization of internal BUFFER resources and low efficiency in short packet processing, thus affecting the overall performance.

[0004] Traditional WQE scheduling strategies usually perform data transmission based on a fixed number of WQE acquisitions, without considering the impact of changes in the number of QPs on performance. When the number of QPs is large, this strategy can maintain a high data transmission efficiency; however, when the number of QPs is small, the data transmission efficiency will drop significantly because the internal BUFFER resources are not fully utilized. Summary of the invention

[0005] In view of this, the present invention provides a WQE adaptive scheduling system and method to solve the problem that the traditional WQE scheduling strategy usually performs data transmission based on a fixed WQE acquisition number without considering the impact of changes in the number of QPs on performance.

[0006] In a first aspect, the present invention provides a WQE adaptive scheduling system, the system comprising a host, a PCIE module, a DB module and a WQE_PROC module; the host is respectively connected to the DB module and the WQE_PROC module through the PCIE module; the DB module is also interconnected with the WQE_PROC module;

[0007] The host is used to generate doorbell information and send the doorbell information to the DB module through the PCIE module;

[0008] The PCIE module is used to establish a data transmission channel between the host and the DB module, and between the host and the WQE_PROC module;

[0009] The DB module is used to receive and save the doorbell information, calculate the number of QP doorbells in the chain based on the doorbell information, and output the number of QP doorbells in the chain to the WQE_PROC module;

[0010] The WQE_PROC module is used to calculate the actual number of QP doorbells based on the number of QP doorbells in the chain, and dynamically adjust the number of WQE acquisitions requested from the host according to the actual number of QP doorbells.

[0011] The above scheme realizes the collaborative work between modules and improves the data transmission efficiency and system response speed by constructing a WQE adaptive scheduling system including the host, PCIE module, DB module and WQE_PROC module. The host generates and sends doorbell information, which can respond to external events or requests in a timely manner and provide trigger signals for WQE scheduling; the PCIE module establishes a data transmission channel to ensure that the doorbell information and WQE data are transmitted quickly and accurately, reducing delays and error rates; the DB module processes the doorbell information, provides key data support for WQE scheduling, and dynamically adjusts the scheduling strategy according to the number of QPs. The WQE_PROC module can flexibly adjust the WQE acquisition strategy according to the actual number of QP doorbells and the system status to optimize system performance.

[0012] In an optional implementation, the WQE_PROC module includes a QP_FIFO component and a LOCK_TABLE component;

[0013] The QP_FIFO component is used to cache and calculate the number of QP doorbells to be processed;

[0014] The LOCK_TABLE component is used to record and calculate the number of QP doorbells being processed.

[0015] The above solution implements efficient management and processing of QP doorbells by introducing QP_FIFO and LOCK_TABLE components in the WQE_PROC module. The QP_FIFO component is responsible for caching and calculating the number of QP doorbells to be processed. At the same time, the LOCK_TABLE component is responsible for recording and calculating the number of QP doorbells being processed. The collaborative work of these two components enables the WQE_PROC module to more accurately grasp the processing status of the QP doorbell, thereby allocating system resources more reasonably and improving the overall performance and stability of the system.

[0016] In an optional implementation, the WQE_PROC module is further configured to add the number of QP doorbells in the chain received from the DB module, the number of QP doorbells to be processed calculated by the QP_FIFO component, and the number of QP doorbells being processed calculated by the LOCK_TABLE component to obtain an actual number of QP doorbells, and when the actual number of QP doorbells is less than a target threshold, request a first WQE acquisition number from the host; when the actual number of QP doorbells is equal to or greater than the target threshold, request a second WQE acquisition number from the host; the first WQE acquisition number is greater than the second WQE acquisition number.

[0017] The above scheme uses the WQE_PROC module to accurately calculate the actual number of QP doorbells, and dynamically adjusts the number of WQE acquisitions requested from the host based on the number. When the actual number of QP doorbells is small, the number of WQE acquisitions is increased to fully utilize system resources and improve processing efficiency; when the actual number of QP doorbells is large, the number of WQE acquisitions is reduced to avoid system overload and resource competition. This dynamic adjustment strategy enables the system to respond flexibly according to different load conditions, ensuring efficient processing and avoiding resource waste, thereby improving the overall performance and stability of the system.

[0018] In an optional implementation, the first WQE acquisition quantity is the number of WQEs stored corresponding to half the size of the BUFFER space in the WQE_PROC module, and the second WQE acquisition quantity is the number of WQEs stored corresponding to one quarter of the BUFFER space in the WQE_PROC module.

[0019] The above scheme sets a specific WQE acquisition quantity strategy, that is, when the actual number of QP doorbells is small, the WQE_PROC module requests the WQE quantity corresponding to half of its BUFFER space (the first WQE acquisition quantity) to make full use of available resources and improve processing efficiency; when the actual number of QP doorbells is large, the WQE quantity corresponding to one-quarter of the BUFFER space is requested (the second WQE acquisition quantity) to avoid system overload and maintain processing stability, so that the system can manage resources more effectively, balance processing efficiency and system stability, and thus improve overall performance.

[0020] In a second aspect, the present invention provides a WQE adaptive scheduling method, which is applied to a WQE_PROC module of a WQE adaptive scheduling system as described above, and the method includes:

[0021] Obtain the number of QP doorbells in the chain generated and sent by the DB module based on the doorbell information; the doorbell information is generated by the host and sent to the DB module through the PCIE module;

[0022] The actual number of QP doorbells is calculated based on the number of QP doorbells in the chain, and the number of WQE acquisitions requested from the host is dynamically adjusted according to the actual number of QP doorbells.

[0023] In an optional implementation manner, the calculating the actual number of QP doorbells based on the number of QP doorbells in the chain includes:

[0024] Obtain the number of QP doorbells to be processed from the QP_FIFO component;

[0025] Obtain the number of QP doorbells to be processed from the LOCK_TABLE component;

[0026] The number of QP doorbells in the chain, the number of QP doorbells to be processed, and the number of QP doorbells being processed are added together to obtain the actual number of QP doorbells.

[0027] In an optional implementation manner, dynamically adjusting the number of WQE acquisitions requested from the host according to the actual number of QP doorbells includes:

[0028] When the actual number of the QP doorbells is less than the target threshold, requesting the host to obtain a first WQE number;

[0029] When the actual number of the QP doorbells is equal to or greater than the target threshold, a second WQE acquisition number is requested from the host; and the first WQE acquisition number is greater than the second WQE acquisition number.

[0030] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements a WQE adaptive scheduling method of the above-mentioned second aspect or any corresponding embodiment thereof by executing the computer instructions.

[0031] In the fourth and fifth aspects, the present invention provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement a WQE adaptive scheduling method according to the second aspect or any corresponding embodiment thereof.

[0032] In a fifth aspect, the present invention provides a computer program product, including computer instructions, which, when executed by a processor, implement a WQE adaptive scheduling method according to the second aspect or any corresponding implementation manner thereof.

[0033] The technical solution provided by the present invention may include the following beneficial effects:

[0034] The number of WQEs acquired each time is dynamically adjusted based on the actual number of QP doorbells. In the scenario of short packets and few QPs, the number of WQEs acquired each time can be dynamically increased to effectively improve the MPPS (Million Packets Per Second) performance, and make full use of the internal BUFFER (buffer) space, so that more short packets can be processed and sent under one scheduling, thereby improving the efficiency of the system in processing short packets and meeting the requirements of application scenarios with high requirements for short packet processing performance. The adaptive WQE scheduling strategy of the present invention can reasonably allocate internal BUFFER resources according to the actual number of QP doorbells. When the actual number of QP doorbells is small, more WQEs are acquired to fully fill the BUFFER to avoid idle waste of resources; and when the actual number of QP doorbells is large, excessive acquisition of WQEs is avoided to cause BUFFER congestion, thereby achieving efficient use of resources and improving the overall performance of the system.

[0035] While improving the performance of short packet and few QP scenarios, it does not affect the system performance in other scenarios (such as long packet scenarios, multi-QP scenarios, etc.). It ensures that the system can run stably under various workloads and network traffic conditions, has good compatibility, and is suitable for complex and changeable InfiniBand network application environments. It can automatically switch the WQE acquisition number according to the actual number of QP doorbells during hardware execution without manual intervention. This automated adaptive mechanism enables the system to respond to network load changes in real time and flexibly adjust scheduling strategies, reducing the complexity and error rate of manual configuration and improving the reliability and maintainability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 is a schematic diagram of internal BUFFER usage corresponding to different numbers of QPs according to an embodiment of the present invention;

[0038] Figure 2 is a structural schematic diagram of a WQE adaptive scheduling system according to an embodiment of the present invention;

[0039] Figure 3 is a flow chart of a WQE adaptive scheduling method according to an embodiment of the present invention;

[0040] Figure 4 is a structural block diagram of a WQE adaptive scheduling device according to an embodiment of the present invention;

[0041] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0043] It should be noted that some key terms in the present invention are first explained:

[0044] WQE: WQE (Work Queue Element) is a data structure used to define operations in the InfiniBand work queue, such as data sending, receiving, atomic operations, etc. It is the core mechanism for InfiniBand to implement data transmission and operations. Each WQE contains key information such as the opcode, buffer pointer, and length field. The opcode specifies the type of operation to be performed, the buffer pointer points to the memory location where the data is stored, and the length field specifies the data length.

[0045] QP: QP is a logical entity used to send and receive data in InfiniBand communication, which consists of a send queue and a receive queue. An application initiates a data transmission operation by placing a WQE into the corresponding work queue. Different applications or data transmission tasks can correspond to different QPs.

[0046] Doorbell module: The doorbell module is a component in the InfiniBand architecture, which is used to notify the system of new tasks or events. When the application puts a WQE into the work queue, it sends a signal to the system through the doorbell module (similar to pressing the doorbell), informing the system that there is a new WQE waiting to be processed. The system performs subsequent WQE scheduling operations based on the doorbell information.

[0047] MPPS (Million Packets Per Second): MPPS is a performance indicator that measures the ability of network equipment to process short packets, indicating the number of million data packets that can be processed per second. In InfiniBand, in short packet scenarios (such as transmission of control information with a small amount of data), the higher the MPPS, the higher the efficiency of the system in processing short packets.

[0048] Bandwidth: Bandwidth refers to the amount of data that can be transmitted per unit time, usually expressed in bytes per second (such as GB / s). In InfiniBand, in long packet scenarios (such as large data file transfers), the higher the bandwidth, the stronger the system's ability to transmit long packet data.

[0049] PCIE interface: PCIE interface is a high-speed serial computer expansion bus standard used for data transmission between different components in a computer system (such as CPU, memory, graphics card, etc.). In the InfiniBand architecture, the PCIE interface is used to implement data interaction between the host and the InfiniBand device (such as a network card), including operations such as obtaining WQE data.

[0050] In the InfiniBand protocol, WQE is the core mechanism for implementing data transmission and operation. WQE defines specific tasks for network operations, such as data sending, receiving, and various memory operations. WQE scheduling is an important part of the InfiniBand implementation. WQE scheduling refers to arranging the execution order of multiple WQEs, with the goal of efficient task processing and system performance optimization. The optimization of WQE scheduling not only affects the system's performance, throughput, and response time, but also is related to important aspects such as the effective use of resources and task priority management.

[0051] In the WQE scheduling process, it is necessary to obtain WQE data from the host through the PCIE interface according to the doorbell information pressed by the doorbell module. The number of WQEs and packet length obtained each time have a great impact on the scheduling performance. There are two standards for measuring performance: MPPS for short packets and bandwidth for long packets.

[0052] 1. When there are many QPs, in the long packet scenario, a single QP obtains a large number of WQEs, and the number of QPs that occupy the internal BUFFER and process at the same time is small. Since all the obtained WQEs cannot be sent out, the bandwidth is lower.

[0053] 2. When the number of QPs is large, in the short packet scenario, the number of WQEs obtained by a single QP has little effect on the MPPS. This is because the obtained WQEs can be sent out, and the actual number of WQEs sent out is the number of WQEs stored in the internal BUFFER.

[0054] 3. When the number of QPs is small, in the long packet scenario, the number of WQEs obtained by a single QP is large or small, which has little impact on the bandwidth because the obtained WQEs cannot be sent out.

[0055] 4. When the number of QPs is small, in the short packet scenario, a single QP obtains a large number of WQEs, and the number of WQEs in the internal BUFFER is large. Since all the obtained WQEs can be sent out, the MPPS is higher.

[0056] It can be seen that in the traditional WQE scheduling strategy, since it is uncertain whether the WQE to be obtained is a long packet or a short packet, and the number of current QPs is also uncertain, the number of WQEs obtained in each scheduling can only be used, which leads to large differences in performance in different scenarios. For example: in the short packet scenario, when the internal BUFFER can cache 128 WQEs, when the number of WQEs is set to 32 and the number of QPs is greater than or equal to 4, the internal BUFFER can be full and the MPPS is high. When the number of QPs is less than 4, the internal BUFFER will be idle, reducing the MPPS.

[0057] The present invention aims to solve the problem that the performance of InfiniBand's WQE scheduling varies greatly in different scenarios, mainly targeting the situation where the MPPS performance needs to be improved in the scenario of short packets and few QPs. The core concept is to design a dynamic scheduling strategy that can perceive the system status in real time and adaptively adjust the number of WQEs obtained to achieve system performance optimization. Based on the above problems, the present invention accurately counts the number of QPs sent to the hardware (including those being processed and not yet processed), and dynamically adjusts the number of WQEs obtained each time according to the changes in the number of QPs. Please refer to Figure 1 The following is a schematic diagram of the internal BUFFER usage corresponding to different numbers of QPs, as shown in Figure 1As shown in the figure, when the number of QPs obtained by statistics (i.e., the actual number of QP doorbells) is less than the target threshold (the target threshold range can be 2-4, 4 in the example), the system is determined to be in a low-QP scenario. At this time, the number of WQEs obtained in a batch of scheduling is set to the number of WQEs corresponding to half of the size of the BUFFER space for storing WQEs in the WQE_PROC module, which is 64 in this example, so as to make full use of the internal BUFFER space, improve the efficiency of short packet processing, and thus improve the MPPS performance; when the number of QPs is greater than or equal to the target threshold, it indicates that the system is in other scenarios (such as multi-QP scenarios or long packet scenarios, etc.), then the number of WQEs obtained is automatically adjusted to the number of WQEs corresponding to one-fourth of the BUFFER space for storing WQEs in WQE_PROC, which is 32 in this example, to avoid excessive occupation of internal BUFFER resources by a single QP, so that the number of QPs processed simultaneously is small, and the system can maintain stable and efficient operation in different scenarios. This adaptive adjustment mechanism can automatically switch seamlessly during hardware execution without manual intervention. The system can respond to factors such as changes in network load and fluctuations in the number of QPs in real time, and flexibly adjust the number of WQEs obtained to ensure that the system performance in short packet and few QP scenarios is significantly improved without affecting the performance of other scenarios, thereby optimizing the overall performance and meeting the needs of diverse network application scenarios.

[0058] In this embodiment, a WQE adaptive scheduling system is provided. Figure 2 is a schematic diagram of the structure of a WQE adaptive scheduling system according to an embodiment of the present invention. Figure 2 As shown, the system includes a host, a PCIE module, a DB module (i.e. Figure 2 The host is connected to the DB module and the WQE_PROC module respectively through the PCIE module; the DB module is also connected to the WQE_PROC module;

[0059] The host is used to generate doorbell information and send the doorbell information to the DB module through the PCIE module;

[0060] The PCIE module is used to establish a data transmission channel between the host and the DB module, and between the host and the WQE_PROC module;

[0061] The DB module is used to receive and store the doorbell information, calculate the number of QP doorbells in the chain based on the doorbell information, and output the number of QP doorbells in the chain to the WQE_PROC module;

[0062] The WQE_PROC module is used to calculate the actual number of QP doorbells based on the number of QP doorbells in the chain (i.e. Figure 2The QP_NUM in the QP_NUM) is used, and the number of WQE acquisitions requested from the host is dynamically adjusted according to the actual number of the QP doorbells.

[0063] Furthermore, the host, PCIE module, DB module and WQE_PROC module in the system work closely together to complete the WQE scheduling task. As the source of data and instructions, the host sends doorbell information to the DB module through the PCIE module. The PCIE module is responsible for establishing a high-speed data transmission channel between the host and other modules to ensure that the doorbell information can be transmitted accurately and quickly. The DB module plays a key role in connecting the upper and lower parts of the entire system. On the one hand, it is responsible for receiving and saving the doorbell information from the PCIE module, and on the other hand, it realizes the storage and sorting of doorbell information through the internal efficient linked list structure, which is convenient for subsequent rapid retrieval and processing. At the same time, the DB module can accurately calculate the number of QP doorbells in the chain and output the data to the WQE_PROC module.

[0064] In an optional embodiment, the WQE_PROC module includes a QP_FIFO component and a LOCK_TABLE component;

[0065] The QP_FIFO component is used to cache and calculate the number of QP doorbells to be processed;

[0066] The LOCK_TABLE component is used to record and calculate the number of QP doorbells being processed.

[0067] In an optional implementation, the WQE_PROC module is further configured to add the number of QP doorbells in the chain received from the DB module, the number of QP doorbells to be processed calculated by the QP_FIFO component, and the number of QP doorbells being processed calculated by the LOCK_TABLE component to obtain an actual number of QP doorbells, and when the actual number of QP doorbells is less than a target threshold, request a first WQE acquisition number from the host, and when the actual number of QP doorbells is equal to or greater than the target threshold, request a second WQE acquisition number from the host; the first WQE acquisition number is greater than the second WQE acquisition number.

[0068] Furthermore, the WQE_PROC module contains two important components, the QP_FIFO component and the LOCK_TABLE component. The QP_FIFO component is mainly used to cache the number of QP doorbells to be processed and calculate the number of QP doorbells to be processed, while the LOCK_TABLE component is responsible for recording the number of QP doorbells being processed and calculating the number of QP doorbells being processed. The WQE_PROC module accurately calculates the actual number of QPs in the current hardware system (i.e., the actual number of QP doorbells) by adding the three data of the number of QP doorbells to be processed in the QP_FIFO component, the number of QP doorbells being processed in the LOCK_TABLE component, and the number of QP doorbells in the chain received from the DB module, providing a key basis for the subsequent adaptive adjustment of the WQE acquisition quantity.

[0069] In an optional implementation, the first WQE acquisition quantity is the WQE quantity stored corresponding to half the size of the BUFFER space in the WQE_PROC module, and the second WQE acquisition quantity is the WQE quantity stored corresponding to one quarter of the BUFFER space in the WQE_PROC module.

[0070] Further, when the WQE acquisition number is adaptively determined, in this embodiment, when the WQE_PROC module acquires the determined WQE acquisition number, the WQE acquisition number is dynamically set according to the current QP number (i.e., the actual number of QP doorbells) calculated by the WQE_PROC module. When the actual number of QP doorbells is less than the target threshold (exemplarily, the target threshold is 4), it indicates that the system is in a low QP scenario. At this time, each QP doorbell scheduling will request the host for the first WQE acquisition number (exemplarily, the first WQE acquisition number is 64) to fully utilize the internal BUFFER resources and improve the short packet processing efficiency; when the number of QPs is greater than or equal to 4, the system is in other scenarios, and the WQE_PROC module automatically adjusts to request the second WQE acquisition number (exemplarily, the second WQE acquisition number is 32), that is, each QP doorbell scheduling will request the host for the second WQE acquisition number to avoid excessive occupation of internal BUFFER resources by a single QP, thereby ensuring the stable operation of the system in different scenarios. That is to say, in this embodiment, each QP doorbell in the actual number of QP doorbells will request the corresponding WQE acquisition number from the host. If the actual number of QP doorbells is less than 4, each QP doorbell will request 64 WQE acquisition numbers from the host, but the total WQE acquisition number corresponding to the actual number of QP doorbells must be less than or equal to the internal BUFFER space (exemplarily, the internal BUFFER space is 128X wqe size, that is, 128×WQE size), that is, the internal BUFFER space caches two QP doorbells at a time, and the other QP doorbells in the actual number of QP doorbells wait for subsequent caching. Similarly, if the actual number of QP doorbells is equal to or greater than 4, each QP doorbell will request 32 WQE acquisition numbers from the host, but the total WQE acquisition number corresponding to the actual number of QP doorbells must be less than or equal to the internal BUFFER space (exemplarily, the internal BUFFER space is 128Xwqe size). size, that is, 128×WQE size), that is, the internal BUFFER space caches four QP doorbells each time, and the other QP doorbells in the actual number of QP doorbells wait for subsequent cache processing. In actual applications, the host first generates doorbell information and sends it to the DB module through the PCIE module. After receiving the doorbell information, the DB module stores it in a linked list and calculates the current number of QP doorbells in the chain. The DB module outputs the calculated number of QP doorbells in the chain to the WQE_PROC module. The QP_FIFO component in the WQE_PROC module caches the doorbell information of the DB module scheduling exit, and the LOCK_TABLE component records the doorbell information being processed. The number of QP doorbells to be processed cached by the QP_FIFO component, the number of QP doorbells being processed recorded in the LOCK_TABLE component, and the number of QP doorbells in the chain input by the DB module are added to obtain the current actual number of QPs (that is, the actual number of QP doorbells).The WQE_PROC module determines the current scenario based on the calculated actual number of QP doorbells. If the actual number of QP doorbells is less than 4, a request is sent to the host to obtain 64 WQEs; if the actual number of QP doorbells is greater than or equal to 4, a request is sent to obtain 32 WQEs. Finally, based on the request of the WQE_PROC module, the host sends the corresponding number of WQE acquisitions to the WQE_PROC module through the PCIE module. The WQE_PROC module receives and processes this data to complete a WQE scheduling process. The system then continuously monitors changes in the number of QPs and repeats the above steps to achieve dynamic adaptive scheduling.

[0071] The following is an explanation of the contents disclosed in the above embodiment through simple examples:

[0072] Assume that the internal processing delay is 16ns (internal processing delay means the time required for the system to process the internal logic of a data packet), the memory read delay is 1500ns (memory read delay means the time required to read a data packet from the memory), and the internal BUFFER size is 128X wqe size (i.e. 128×WQE size). The total scheduling time is 1500+16×wqe number (ns). It can be seen that the scheduling time includes the time required to read the data packet from the memory and process each WQE. The comparison of the number of cached WQEs and the packet sending rate of the adaptive strategy of this embodiment and the related fixed scheme under different QP numbers is shown in Table 1.

[0073] Table 1

[0074]

[0075] As can be seen from Table 1, for the number of WQEs cached by the adaptive policy in this embodiment: when the number of QPs (i.e., the actual number of QP doorbells) is 1, the number of WQEs cached by the adaptive policy (i.e., the number of WQEs obtained) is 64, and when the number of QPs is 2 and 3, the number of WQEs cached by the adaptive policy is 128. This is because in the scenario of short packets and few QPs, this embodiment dynamically adjusts the number of WQEs obtained according to the number of QPs, and when the number of QPs is less than the target threshold (range 2-4, 4 in the example), it is set to 64 WQEs (the number of WQEs that can be stored in half of the BUFFER space), in order to make full use of the internal BUFFER space (the size is 128X wqe size) so that the system can process more short packets in one scheduling. The number of cached WQEs for the relevant fixed scheme: the number of cached WQEs under different QP numbers is 32 (when the number of QP is 1), 64 (when the number of QP is 2) and 96 (when the number of QP is 3). The number of WQEs obtained does not adjust with the change of the number of QPs and is relatively fixed.

[0076] In the adaptive strategy of this embodiment, when the number of QPs is 1, the packet rate is 25.36Mpps; when the number of QPs is 2 and 3, the packet rate is 36.09Mpps. As the number of WQEs obtained increases, the number of short packets that can be processed and sent increases when the scheduling time increases relatively little, thereby improving the packet rate. For example, when the number of QPs is 1, the time for the adaptive strategy of this embodiment to process 64 WQEs is 1500+16×64=2524ns, and the number of data packets that can be processed per second (packet rate) is 1000000000 / 2524≈25.36Mpps. The packet rates of related fixed schemes at different QP numbers are relatively low, such as when the number of QPs is 1, the packet rate is 15.91Mpps, when the number of QPs is 2, the packet rate is 25.36Mpps, and when the number of QPs is 3, the packet rate is 31.63Mpps. Because the number of WQEs obtained cannot be optimized and adjusted according to the number of QPs, the internal BUFFER space cannot be fully utilized, resulting in a relatively small number of short packets processed in one scheduling and a limited packet sending rate.

[0077] In summary, this embodiment dynamically adjusts the number of WQEs acquired each time based on the actual number of QP doorbells. In the scenario of short packets and few QPs, by dynamically increasing the number of WQEs acquired each time, the MPPS (Million Packets Per Second) performance can be effectively improved, and the internal BUFFER (buffer) space can be fully utilized, so that more short packets can be processed and sent under one scheduling, thereby improving the efficiency of the system in processing short packets and meeting the requirements of application scenarios with high requirements for short packet processing performance. The adaptive WQE scheduling strategy of this embodiment can reasonably allocate internal BUFFER resources according to the actual number of QP doorbells. When the actual number of QP doorbells is small, more WQEs are acquired to fully fill the BUFFER to avoid idle waste of resources; and when the actual number of QP doorbells is large, excessive acquisition of WQEs is avoided to cause BUFFER congestion, thereby achieving efficient use of resources and improving the overall performance of the system.

[0078] This embodiment improves the performance of the short packet and few QP scenarios without affecting the system performance in other scenarios (such as long packet scenarios, multi-QP scenarios, etc.). It ensures that the system can run stably under various workloads and network traffic conditions, has good compatibility, and is suitable for complex and changeable InfiniBand network application environments. It can automatically switch the WQE acquisition number according to the actual number of QP doorbells during the hardware execution process without manual intervention. This automated adaptive mechanism enables the system to respond to network load changes in real time, flexibly adjust scheduling strategies, reduce the complexity and error rate of manual configuration, and improve the reliability and maintainability of the system.

[0079] According to an embodiment of the present invention, a WQE adaptive scheduling method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0080] In this embodiment, a WQE adaptive scheduling method is provided, which is applied to Figure 2 In the WQE_PROC module of a WQE adaptive scheduling system shown, Figure 3 is a flow chart of a WQE adaptive scheduling method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0081] Step S301, obtaining the number of QP doorbells in the chain generated and sent by the DB module based on the doorbell information; the doorbell information is generated by the host and sent to the DB module through the PCIE module.

[0082] Furthermore, the host, as the starting point of the entire scheduling process, generates doorbell information. The doorbell information is accurately and quickly sent to the DB module through the high-speed data transmission channel of the PCIE module. After receiving the doorbell information, the DB module uses its internal efficient linked list structure for storage and sorting, and based on this, calculates the number of QP doorbells in the chain, and then outputs the data to the WQE_PROC module. This step establishes a data transmission link from the host to the DB module and then to the WQE_PROC module, providing basic data for the subsequent calculation of the actual number of QP doorbells. The number of QP doorbells in the chain reflects the number of tasks related to QP that are currently waiting to be processed or being processed in the link, and is an important basis for the entire scheduling decision-making process. Its accuracy enables the system to fully understand the current task load situation, thereby providing a reference for reasonably adjusting the number of WQE acquisitions.

[0083] Step S302: Calculate the actual number of QP doorbells based on the number of QP doorbells in the chain, and dynamically adjust the number of WQE acquisitions requested from the host according to the actual number of QP doorbells.

[0084] Furthermore, this embodiment obtains the number of QP doorbells to be processed from the QP_FIFO component. The QP_FIFO component is mainly used to cache and calculate the number of QP doorbells to be processed, which represents the number of QP tasks that have not yet started to be processed but are waiting in line; this embodiment obtains the number of QP doorbells to be processed and being processed from the LOCK_TABLE component. The LOCK_TABLE component is responsible for recording the number of QP doorbells being processed, which reflects the number of QP tasks currently being processed by the system. This embodiment adds the actual number of QP doorbells, the number of QP doorbells to be processed, and the number of QP doorbells being processed, so that the total actual number of QP doorbells in the system can be obtained. The actual number of QP doorbells comprehensively considers the QP tasks in different states in the system and fully reflects the current task load level of the system.

[0085] In an optional implementation, step S302 includes:

[0086] Get the number of QP doorbells to be processed from the QP_FIFO component;

[0087] Get the number of QP doorbells to be processed from the LOCK_TABLE component;

[0088] The number of QP doorbells in the chain, the number of QP doorbells to be processed, and the number of QP doorbells being processed are added to obtain the actual number of QP doorbells;

[0089] When the actual number of the QP doorbells is less than the target threshold, requesting the host to obtain the first WQE number;

[0090] When the actual number of the QP doorbells is equal to or greater than the target threshold, a second WQE acquisition number is requested from the host; and the first WQE acquisition number is greater than the second WQE acquisition number.

[0091] Furthermore, this embodiment sets a target threshold (the target threshold may be 4), and compares the calculated actual number of QP doorbells with the threshold. When the actual number of QP doorbells is less than the target threshold, it indicates that the system is in a low-QP scenario; when the actual number of QP doorbells is equal to or greater than the target threshold, it indicates that the system is in other scenarios (such as a multi-QP scenario or a long packet scenario, etc.). This embodiment requests different numbers of WQEs (i.e., the number of WQE acquisitions) from the host according to the comparison result. When the actual number of QP doorbells is less than the target threshold, a first WQE acquisition number is requested from the host (the first WQE acquisition number is the number of WQEs stored corresponding to half the size of the BUFFER space in the WQE_PROC module. For example, if the BUFFER space is 128X wqesize, that is, 128×WQE size, the first WQE acquisition number can be 64). By acquiring more WQEs, the internal BUFFER space is fully utilized to improve the short packet processing efficiency to meet the needs of improving MPPS performance in a scenario with few QPs; when the actual number of QP doorbells is equal to or greater than the target threshold, a second WQE acquisition number is requested from the host (the second WQE acquisition number is the number of WQEs stored corresponding to one-quarter the size of the BUFFER space in the WQE_PROC module. For example, if the BUFFER space is 128X wqesize, that is, 128×WQE size, the first WQE acquisition number can be 64). size, that is, 128×WQE size, then the second WQE acquisition quantity can be 32), to avoid excessive occupation of internal BUFFER resources by a single QP, and ensure the stable operation of the system in other scenarios. This dynamic adjustment mechanism can flexibly optimize the WQE acquisition strategy according to the actual state of the system and improve the overall performance of the system.

[0092] In summary, this embodiment dynamically adjusts the number of WQEs acquired each time based on the actual number of QP doorbells. In the scenario of short packets and few QPs, by dynamically increasing the number of WQEs acquired each time, the MPPS (Million Packets Per Second) performance can be effectively improved, and the internal BUFFER (buffer) space can be fully utilized, so that more short packets can be processed and sent under one scheduling, thereby improving the efficiency of the system in processing short packets and meeting the requirements of application scenarios with high requirements for short packet processing performance. The adaptive WQE scheduling strategy of this embodiment can reasonably allocate internal BUFFER resources according to the actual number of QP doorbells. When the actual number of QP doorbells is small, more WQEs are acquired to fully fill the BUFFER to avoid idle waste of resources; and when the actual number of QP doorbells is large, excessive acquisition of WQEs is avoided to cause BUFFER congestion, thereby achieving efficient use of resources and improving the overall performance of the system.

[0093] This embodiment improves the performance of the short packet and few QP scenarios without affecting the system performance in other scenarios (such as long packet scenarios, multi-QP scenarios, etc.). It ensures that the system can run stably under various workloads and network traffic conditions, has good compatibility, and is suitable for complex and changeable InfiniBand network application environments. It can automatically switch the WQE acquisition number according to the actual number of QP doorbells during the hardware execution process without manual intervention. This automated adaptive mechanism enables the system to respond to network load changes in real time, flexibly adjust scheduling strategies, reduce the complexity and error rate of manual configuration, and improve the reliability and maintainability of the system.

[0094] In this embodiment, a WQE adaptive scheduling device is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0095] This embodiment provides a WQE adaptive scheduling device, which is applied to Figure 2 In the WQE_PROC module of a WQE adaptive scheduling system shown in FIG. Figure 4 As shown, including:

[0096] A QP doorbell chain number generation module 401 is used to obtain the QP doorbell chain number generated and sent by the DB module based on the doorbell information; the doorbell information is generated by the host and sent to the DB module through the PCIE module;

[0097] The WQE acquisition quantity adjustment module 402 is used to calculate the actual number of QP doorbells based on the number of QP doorbells in the chain, and dynamically adjust the WQE acquisition quantity requested from the host according to the actual number of QP doorbells.

[0098] In some optional implementations, the QP doorbell in-chain number generating module 401 is further used to:

[0099] Get the number of QP doorbells to be processed from the QP_FIFO component;

[0100] Get the number of QP doorbells to be processed from the LOCK_TABLE component;

[0101] The number of QP doorbells in the chain, the number of QP doorbells to be processed, and the number of QP doorbells being processed are added together to obtain the actual number of QP doorbells.

[0102] In some optional implementations, the WQE acquisition quantity adjustment module 402 is further configured to:

[0103] When the actual number of the QP doorbells is less than the target threshold, requesting the host to obtain the first WQE number;

[0104] When the actual number of the QP doorbells is equal to or greater than the target threshold, a second WQE acquisition number is requested from the host; and the first WQE acquisition number is greater than the second WQE acquisition number.

[0105] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0106] In this embodiment, a WQE adaptive scheduling device is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0107] The embodiment of the present invention also provides a computer device, see Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0108] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0109] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0110] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0111] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0112] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0113] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0114] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0115] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the defined scope.

Claims

1. A WQE adaptive scheduling system, characterized in that: The system includes a host, a PCIE module, a DB module and a WQE_PROC module; the host is respectively connected to the DB module and the WQE_PROC module through the PCIE module; the DB module is also interconnected with the WQE_PROC module; The host is used to generate doorbell information and send the doorbell information to the DB module through the PCIE module; The PCIE module is used to establish a data transmission channel between the host and the DB module, and between the host and the WQE_PROC module; The DB module is used to receive and save the doorbell information, calculate the number of QP doorbells in the chain based on the doorbell information, and output the number of QP doorbells in the chain to the WQE_PROC module; The WQE_PROC module is used to calculate the actual number of QP doorbells based on the number of QP doorbells in the chain, and dynamically adjust the number of WQE acquisitions requested from the host according to the actual number of QP doorbells.

2. The system according to claim 1, characterized in that The WQE_PROC module includes a QP_FIFO component and a LOCK_TABLE component; The QP_FIFO component is used to cache and calculate the number of QP doorbells to be processed; The LOCK_TABLE component is used to record and calculate the number of QP doorbells being processed.

3. The system according to claim 2, characterized in that The WQE_PROC module is further configured to add the number of QP doorbells in the chain received from the DB module, the number of QP doorbells to be processed calculated by the QP_FIFO component, and the number of QP doorbells being processed calculated by the LOCK_TABLE component to obtain an actual number of QP doorbells, and request a first WQE acquisition number from the host when the actual number of QP doorbells is less than a target threshold, and request a second WQE acquisition number from the host when the actual number of QP doorbells is equal to or greater than the target threshold; the first WQE acquisition number is greater than the second WQE acquisition number.

4. The system according to claim 3, characterized in that The first WQE acquisition quantity is the number of WQEs stored corresponding to half the size of the BUFFER space in the WQE_PROC module, and the second WQE acquisition quantity is the number of WQEs stored corresponding to one quarter of the BUFFER space in the WQE_PROC module.

5. A WQE adaptive scheduling method, characterized in that: The method is applied to a WQE_PROC module in a WQE adaptive scheduling system according to any one of claims 1 to 4, and the method comprises: Obtain the number of QP doorbells in the chain generated and sent by the DB module based on the doorbell information; the doorbell information is generated by the host and sent to the DB module through the PCIE module; The actual number of QP doorbells is calculated based on the number of QP doorbells in the chain, and the number of WQE acquisitions requested from the host is dynamically adjusted according to the actual number of QP doorbells.

6. The method according to claim 5, characterized in that The WQE_PROC module includes a QP_FIFO component and a LOCK_TABLE component. The actual number of QP doorbells is calculated based on the number of QP doorbells in the chain, including: Obtain the number of QP doorbells to be processed from the QP_FIFO component; Obtain the number of QP doorbells to be processed from the LOCK_TABLE component; The number of QP doorbells in the chain, the number of QP doorbells to be processed, and the number of QP doorbells being processed are added together to obtain the actual number of QP doorbells.

7. The method according to claim 5, characterized in that The dynamically adjusting the number of WQE acquisitions requested from the host according to the actual number of the QP doorbells includes: When the actual number of the QP doorbells is less than the target threshold, requesting the host to obtain a first WQE number; When the actual number of the QP doorbells is equal to or greater than the target threshold, a second WQE acquisition number is requested from the host; and the first WQE acquisition number is greater than the second WQE acquisition number.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements a WQE adaptive scheduling method according to any one of claims 5 to 7 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, a WQE adaptive scheduling method according to any one of claims 5 to 7 is implemented.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement a WQE adaptive scheduling method according to any one of claims 5 to 7.

Images