Extensible efficient scheduling method suitable for RDMA network mixed message scene

Abstract

The invention provides an extensible efficient scheduling method suitable for an RDMA network mixed message scene, and belongs to the technical field of computer network system flow control. The method comprises the following steps: S1, designing a priority scheduler: dynamically distinguishing small messages from tiny messages according to the load size of a working queue element, and allocating higher priorities to the small messages to maximize the utilization rate of a data path and reduce the additional delay of a control path caused by cache miss; s2, integral limiter design: adopting a heuristic current limiting strategy for the tiny information, and reducing extra delay of a control path caused by frequent cache miss of the tiny information; and S3, designing a sub-message polling device: pre-generating sub-completion queue elements for small messages and micro messages, ensuring that an application program schedules working queue elements at a proper rate, and fully utilizing a sending queue to relieve excessive queuing delay and reduce the frequency of cache miss of the working queue elements. Compared with the prior art, the method effectively solves the problems that the throughput of the RDMA network of the data center is severely reduced and 99% tail time delay of the message is greatly increased in a high-concurrency scene of the message.

Description

Technical Field

[0001] The present invention relates to the technical field of computer network system flow control, and in particular to an extensible and efficient scheduling method suitable for RDMA (remote direct memory access) network hybrid message scenarios. Background Art

[0002] Remote Direct Memory Access (RDMA) technology has been widely adopted in modern data centers due to its kernel bypass, zero copy and other features, replacing the TCP protocol to improve network performance. Many existing applications (such as remote procedure calls, web search engines, and key-value database storage) rely heavily on RDMA to achieve high throughput and low latency as well as low CPU overhead. However, due to limited on-chip resources, there is still a large gap between the IOPS supported by the RDMA network interface card (RDMA NIC, abbreviated as RNIC) and the large number of concurrent remote accesses required by applications, which can lead to a significant drop in throughput (up to 50%) for applications with too many concurrent operations, which is known as the scalability problem of RDMA.

[0003] In recent years, the academic community has proposed many solutions to the scalability problem. These solutions can be divided into two categories (such as Figure 1 shown):

[0004] One is that there are too many concurrent QP (QP is a communication channel between different processes of two hosts in RDMA, which can be understood as data packets transmitted to the other host through QP) connections at the application layer, resulting in too many QP Contexts (which can be understood as the basic elements for maintaining RDMA QP connection information) that cannot be stored in the limited on-chip cache of the network card (the cache for storing QPC is called ICMCache), resulting in frequent cache misses. This will cause the network card to frequently obtain information from the host memory through PCIe (each time there will be a delay of several microseconds), resulting in a significant drop in throughput.

[0005] Another type is that due to the excessive number of concurrent WQE messages within a single QP connection at the application layer (which can be called the "concurrency depth" of QP), the WQE Context cannot be stored in the limited on-chip cache of the network card (the cache that stores WQE is called WQECache), resulting in frequent cache misses, and then causing frequent PCIe to obtain host memory, resulting in a significant drop in throughput.

[0006] In general, the throughput drop caused by too many QPs and too many WQEs in a QP (concurrency depth) are both due to too many concurrently executed RDMA operations. The industry has made good progress in solving the throughput drop caused by too many QPs, enabling a single RDMA network card to support more than 10,000 concurrent QP connections without reducing throughput (the concurrency depth is 1 at this time).

[0007] However, the throughput drop caused by too many WQEs (concurrency depth) in each QP did not achieve good results (described in detail below). When the number of concurrent QPs is small, an appropriate concurrency depth can improve the parallelism of RDMA operations and thus improve network throughput. However, the application load changes dynamically. When the number of concurrent QPs is thousands or even tens of thousands, even if the concurrency depth of each QP is very small, it will cause serious damage to the performance of the RDMA network card. In order to solve the above problems, the present invention proposes a scalable and efficient scheduling method suitable for RDMA network hybrid message scenarios. Summary of the invention

[0008] The purpose of the present invention is to propose a scalable and efficient scheduling method suitable for RDMA network hybrid message scenarios to solve the problems of severe throughput degradation and significant increase in 99% tail latency of messages encountered by data center RDMA (remote direct memory access) networks in high message concurrency scenarios.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A scalable and efficient scheduling method suitable for RDMA network mixed message scenarios, including the following:

[0011] S1. Priority Scheduler design: Dynamically distinguish small messages from tiny messages according to the load size of the work queue element (WQE), and assign higher priority to small messages to maximize the utilization of the data path and reduce the additional delay of the control path caused by cache misses;

[0012] S2. Priority Scheduler design: adopts a heuristic current limiting strategy for small information to reduce the additional delay of the control path caused by frequent cache misses of small information;

[0013] S3. Sub-message Poller design: Pre-generate sub-completion queue elements (CQE) for small and tiny messages to ensure that applications schedule work queue elements (WQE) at an appropriate rate and make full use of the send queue to alleviate excessive queuing delays and reduce the frequency of work queue element (WQE) cache misses.

[0014] Preferably, the S1 specifically includes the following contents:

[0015] The Quality of Service (QoS) mechanism is used to assign higher priority to small messages to improve the utilization of the data path. Specifically:

[0016] Distinguish tiny information from small information based on the length field carried by the work queue element (WQE); map the QP corresponding to the small information to the highest priority flow controller by modifying the DSCP field in the IP header of the data packet; select the enhanced transmission selection (ETS) strategy to allocate higher bandwidth to the flow controller (TC) designated for small information; after the high-priority flow controller (TC) has completed its service, allocate the remaining bandwidth to the flow controller (TC) corresponding to the small information.

[0017] Preferably, S2 specifically includes the following contents:

[0018] The existing Smart algorithm is designed and adjusted to control the tiny messages in the RNIC and reduce the cache misses of the tiny messages. For small messages, a sub-message poller is used to replace the credit-based restriction strategy. The cycle model of the Smart algorithm is retained, and each cycle is divided into an update phase and a stable phase. In the stable phase, a credit system is implemented for each QP that processes tiny messages, and the credit in the credit system represents the maximum number of work queue elements (WQEs) continuously sent by the QP. As the number of work queue elements (WQEs) sent by each QP increases, the credit decreases accordingly. When the credit is exhausted to zero, the QP stops sending new work queue elements (WQEs) and polls its corresponding CQ to see if there is a completion queue element (CQE). After polling the completion queue element (CQE), the credit increases according to the number of unsignaled completions of the work queue element (WQE) on the QP.

[0019] Preferably, S3 specifically includes the following contents:

[0020] The sub-message poller is used to dynamically determine the number of unsignaled completions with the best system performance, and is specifically designed as follows:

[0021] The send_flags of the work queue element (WQE) is modified to allow the work queue element (WQE) originally configured for signal-free completion to generate a completion queue element (CQE) to ensure that the transmission rate and concurrency performance of small messages are not affected and cache misses are reduced; the wr_id field of the work queue element (WQE) is used to indicate whether the work queue element (WQE) is specified as signal-free completion according to the application's settings; when the system calls the ibv_poll_cq API to poll a child completion queue element (CQE), the child completion queue element (CQE) enters the middleware layer between the application and the driver, and all child completion queue elements (CQE) are discarded at this layer to ensure that the application layer is unaware of the above-mentioned additional completion queue elements (CQE).

[0022] A scalable and efficient scheduling system for RDMA network hybrid message scenarios applied to the above method is further proposed. The system is denoted as WQEFC, which is used to establish middleware between the application and the user space driver to ensure the transparency of the two. The system includes:

[0023] The Priority Scheduler module is used to assign different priorities to small and tiny messages;

[0024] The Priority Scheduler module is used to control the concurrency depth of tiny messages.

[0025] Sub-message Poller module, used to pre-generate sub-completion queue elements (CQEs) for small and tiny messages.

[0026] Compared with the prior art, the present invention provides a scalable and efficient scheduling method suitable for RDMA network hybrid message scenarios, which has the following beneficial effects:

[0027] (1) The present invention designs a priority scheduler, which dynamically distinguishes small messages from tiny messages according to the load size of the work queue element, assigns a higher priority to small messages, and can maximize the utilization of the data path, effectively reducing the additional delay of the control path caused by cache misses;

[0028] (2) The present invention designs an integral limiter and adopts a heuristic current limiting strategy for tiny information, which effectively reduces the additional delay of the control path caused by frequent cache misses of tiny information;

[0029] (3) The present invention designs a sub-message poller to pre-generate sub-completion queue elements for small messages and tiny messages, ensuring that the application can schedule work queue elements at an appropriate rate and fully utilize the sending queue, effectively alleviating excessive queuing delays and reducing the frequency of work queue element cache misses.

[0030] In summary, the present invention effectively solves the problem of severe throughput decline and substantial increase in 99% tail latency of messages encountered by the RDMA network in the data center in a high-concurrency message scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 These are the two common technical solutions for solving the scalability problem of RDMA mentioned in the background technology of the present invention;

[0032] Figure 2 It is a structural diagram of the scalable and efficient scheduling system WQEFC applicable to the RDMA network hybrid message scenario mentioned in Example 1 of the present invention;

[0033] Figure 3 This is a model structure diagram after the Smart credit-based restriction strategy mentioned in Example 1 of the present invention is modified;

[0034] Figure 4 A schematic diagram of the structure of the sub-message poller designed in Embodiment 1 of the present invention;

[0035] Figure 5 The throughput of the three methods of the unoptimized general method (i.e., Common), SMART, and WQEFC mentioned in Example 2 of the present invention at different concurrency depths and illegal signal completion counts, where (a) different concurrency depths, number of threads = 64, number of silent completions = 1; (b) number of silent completions, number of threads = 64, concurrency depth = 64; (c) different concurrency depths and silent completions, number of silent completions = concurrency depth, number of threads = 6;

[0036] Figure 6 It is the 99th tail delay of the three methods of the unoptimized general method (i.e. Common), SMART and WQEFC mentioned in Example 2 of the present invention under different concurrency conditions for small messages (≤32B), where (a) different concurrency depths, number of threads = 64, unsignaled completion count = 1; (b) different unsignaled completion counts, number of threads = 64, concurrency depth = 64; (c) different concurrency depths and unsignaled completion counts, unsignaled completion count = concurrency depth, number of threads = 64;

[0037] Figure 7It is the 9th tail delay of small messages (>32B) of the three methods mentioned in Example 2 of the present invention, namely the unoptimized general method (i.e., Common), SMART and WQEFC, under different concurrency conditions, where: (a) different concurrency depths, number of threads = 64, number of unsignaled completions = 1; (b) different unsignaled completion counts, number of threads = 64, concurrency depth = 64; (c) different concurrency depths and unsignaled completion counts, unsignaled completion count = concurrency depth, number of threads = 64. DETAILED DESCRIPTION

[0038] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] First, let's introduce the RDMA (remote direct memory access) workflow. The key terms that will be used later are:

[0040] Taking the read operation as an example, the workflow of RDMA is analyzed. The application encapsulates the message information into a work request (WR) and submits it to the driver layer through the ibv_post_send() API. After the driver layer parses the WR, it converts it into a WQE and writes it into the SQ buffer of the QP. Since the speed at which the application publishes WQEs may exceed the processing capacity of the RNIC, the QP serves as a buffer for WQEs that have not yet been distributed to the RNIC. Subsequently, the RNIC actively obtains and concurrently parses the WQE to be sent from the SQ and encapsulates it into a data packet sent to the receiving host.

[0041] After receiving the ACK signal, the sending host searches the RNIC's WQE cache for necessary information (e.g., wr_id, length) to generate a completion queue element (CQE) indicating the WQE completion status. If the necessary data is not found, a cache miss occurs, which requires the RNIC to query and retrieve the necessary information from the host memory through PCIe (mainly through the control path). Subsequently, the RNIC uses the DMA engine (mainly through the data path) to transfer the data to the corresponding host memory. It is worth noting that WQEs configured with unsignaled completions do not result in the generation of corresponding CQEs. For the remaining WQEs, the RNIC generates corresponding CQEs and writes them to the host memory within the completion queue (CQ). Subsequently, the driver assembles CQEs to generate work completions (WCs). The application determines the frequency of calling the ibv_poll_cq() API to poll the WC based on the configured concurrency depth. For small messages, the data path duration is usually short, which makes the control path a bottleneck for message completion time because cache misses bring an additional overhead of several microseconds for each WQE. As mentioned above, the terms WR and WQE as well as WC and CQE represent the same concept at different protocol layers. Therefore, the present invention will refer to them as WQE and CQE respectively below.

[0042] In the RDMA workflow, concurrency depth and silent completion number are the key factors affecting concurrency performance, which will be analyzed in detail below.

[0043] Concurrency depth: Increasing the concurrency depth can improve parallelism, but it also means that the WQE delivery rate and the RNIC processing rate will become increasingly unbalanced. When the concurrency depth is too high, the number of OWRs in the WQE cache will exceed the cache capacity, resulting in severe cache misses and severe performance degradation.

[0044] Silent completion: By default, all WQEs generate CQEs when they are processed, but in scenarios with high performance requirements, some WQEs may be configured not to generate CQEs, a practice called silent completion. Typically, applications adjust the submission rate of WQEs based on the remaining depth of the SQ to ensure efficient utilization of the SQ and maximize message parallelism. If used properly, unsignaled completions can reduce the number of DMAs and CPU usage of WQE polling, effectively improving network performance. However, excessive use of silent completions can cause applications to misestimate the number of completed WQEs, resulting in reduced submission rates and significant performance degradation at the application level. In addition, in applications with high WQE concurrency depth, carefully controlled silent completions can be strategically adopted to prevent CQ overflow, and this number can be carefully limited to not exceed the concurrency depth. The number of silent completions cannot exceed the value of the concurrency depth.

[0045] Overall, the appropriate level of concurrency depth and number of silent completions are affected by multiple factors, including the current number of concurrent threads and message size. Therefore, dynamically adjusting these parameters as the workload changes is critical to maintaining optimal performance of RDMA.

[0046] Currently, the most advanced solution to solve WQE cache misses on the market is Smart, which uses an integral-based current limiting mechanism to limit the number of concurrent WQEs, thereby significantly improving the throughput of concurrent messages with the same message size. Its algorithm is shown below.

[0047]

[0048] Among them, Credit is the credit. Each QP has its own credit. When the credit is 0, the QP stops sending WQE messages. The credit can only be replenished after the QP polls the CQE corresponding to the WQE.

[0049] SmartPostSend function: The size here refers to whether there is a linked list in a wqe, pointing to the next wqe. The size can be understood as the number of silent completions (a wqe set to silent completion will not generate a cqe. If the silent completion is 4, then the 1st, 2nd, and 3rd wqes will not have a cqe, and only the 4th wqe will have a cqe. That is, only one cqe is generated for every 4 wqes, and the application knows that 4 wqes have been completed when it receives the cqe). The 6th line indicates that a wqe is sent once (for example, if the silent completion number is 4, it can be understood as sending 4 wqes concurrently, because only the 4th wqe needs to wait for the cqe to be polled, and if there is no polling, no new sending will be performed), and the points are subtracted from the corresponding size (indicated). The 5th line indicates that when the credit is 0, stop sending wqe.

[0050] SmartPollCQ function: that is, every time a cqe is received, the points are added to the corresponding size (silent completion number), indicating that the batch of wqes sent have been received

[0051] UpdateCAX and Update functions: Both indicate a heuristic algorithm for periodically updating the credit points of each QP. Since the workload of the application is constantly changing, in order to dynamically keep up with the changes in the application to maintain the maximum throughput of the network, Smart's work has two parts: the stable phase and the update phase. The stable phase is to send and limit wqe based on the points found by the heuristic algorithm in the update phase. This phase does not involve changes in points. The update phase is to try different points for a certain QP one by one in a very short period of time, and record the throughput corresponding to these different points (calculated by the number of cqes received per unit time), find the points corresponding to the maximum throughput, and use them as the points that the QP has been using in the stable phase (each QP changes its own, and different QPs may not be the same).

[0052] Combining the above description and the analysis of the prior art, it can be seen that the Smart algorithm has the following problems: Smart does not consider the blocking of very small messages (less than 32 bytes) on small messages (32 bytes to several thousand bytes) when different messages are mixed (in RDMA, messages can be divided into two parts: a control header and a data payload. The control header goes through the control path, and the data payload goes through the data path. The data payload of very small messages is small, so the processing time in the data path is relatively short. When the cache misses, the control path of the very small message needs to wait for the process of obtaining control information from the host through PCIe, so the time is relatively long. Therefore, for very small messages, a large part of their data path is idle. For small messages (32 bytes to several thousand bytes), the data path time is relatively long, which can cover the longer delay of the cache miss control path. Combined with the experimental results of the present invention, when very small messages are mixed with messages, the 99% tail delay of the small message increases by 2.06 times compared with when not mixed. After analysis, it is believed that this is due to the blocking of small messages by very small messages, which makes the data path unable to be effectively utilized, especially when the cache misses). In addition, when the credits are exhausted, Smart will suspend the WQE transmission of the corresponding QP, regardless of whether the message is a small message (more than 32 bytes) or an extremely small message (less than 32 bytes), thereby introducing additional queuing delays, especially in scenarios with a high number of silent completions (for example, if Smart is within a QP, the credits are exhausted and blocked after sending 4 WQEs. At this time, if the silent completion number is 1, then each time the CQE corresponding to this WQE is polled, the credits can be replenished (credits + 1), and the subsequent ones can continue to be sent. But if the silent completion number is 4, then the CQEs of 4 wqes need to be polled to replenish the credits (credits + 4), and then the subsequent wqes can be sent, which is equivalent to a longer blocking time).

[0053] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0054] Embodiment 1:

[0055] The original technology Smart cannot guarantee the tail delay of these messages well in mixed message scenarios (where both very small messages and small messages exist), especially for small messages, whose 99% tail delay will increase by 2.06 times compared to the non-mixed scenario (blocking of small messages by very small messages; stopping sending WQE messages when points are exhausted, resulting in queuing delays, especially in scenarios with high silent completion counts).

[0056] The tail latency of small messages is important because small messages are very sensitive to latency. Even a slight increase in latency will cause serious degradation of system performance, resulting in unacceptable economic losses. Google reported that due to an increase of 0.5 seconds in search page generation time, search query traffic decreased by 25%. For every 100 milliseconds of delay, Amazon sales will decrease by 1%.

[0057] In order to solve the above problems, the present invention proposes a scalable and efficient scheduling method suitable for RDMA network hybrid message scenarios, and further designs a scalable, low-latency RDMA message scheduler matching this method, denoted as WQEFC, which aims to solve the drag of WQE cache misses on the tail delay of small messages in hybrid message scenarios, and ensure that hybrid messages can maintain low latency and high throughput in high concurrency scenarios.

[0058] The design guiding principle of WQEFC is to schedule messages based on their unique characteristics (extremely small messages do not fully utilize the data path, so performance is greatly affected when there is a cache miss; small messages make relatively full use of the data path, so performance is basically unaffected when there is a cache miss) to ensure that the data path can be fully utilized when messages are mixed, and to reduce the additional delay caused by cache miss to the control path.

[0059] The architecture design of WQEFC is as follows Figure 2 As shown in the figure, it contains three core components. The first component is PriorityScheduler, which is responsible for assigning different priorities to small messages and tiny messages. The second component is Credit Limiter, which is used to control the concurrency depth of tiny messages. The third component is Sub-message Poller, which is responsible for pre-generating sub-CQEs for small messages and tiny messages. The following chapters will introduce these components.

[0060] ① (Priority Scheduler) First, dynamically distinguish small messages from tiny messages according to the WQE load size, and assign higher priority to small messages to maximize the utilization of the data path, thereby reducing the additional delay of the control path caused by cache misses. Specifically, it includes the following:

[0061] In RDMA communication, the control path is responsible for managing the connection state and resources, which is a time-consuming process. For tiny messages, the duration of the data path transmitting data is relatively short, and most of the time is spent waiting for the control path to complete operations (packet header parsing, flow table lookup). In particular, when a cache miss occurs, the time required for the control path will increase significantly, causing the data path to be underutilized and idle for a long time.

[0062] In order to improve the utilization rate of the data path, the present invention assigns a higher priority to small messages through a quality of service (QoS) mechanism. Specifically, micro messages and small messages are distinguished according to the length field carried by the WQE. By modifying the DSCP field in the IP header of the data packet, the QP corresponding to the small message is mapped to the highest priority traffic controller (TC). The present invention adopts an enhanced transmission selection (ETS) strategy to allocate higher bandwidth to the TC designated for small messages. After the high-priority TC is served, the remaining bandwidth is redistributed to the TC corresponding to the small message. Such a design provides a minimum bandwidth guarantee for small messages, avoiding the small messages from being in a hungry state for a long time. At the same time, it can also more effectively utilize idle data paths and use the longer data transmission time of small messages to make up for the control path overhead of small messages.

[0063] ② (Credit, Limiter) Secondly, the present invention adopts a heuristic current limiting strategy for small messages to reduce the additional delay of the control path caused by frequent cache misses of small messages, which specifically includes the following contents:

[0064] As mentioned above, tiny messages are very sensitive to cache misses. Therefore, the present invention adjusts and slightly modifies the Smart credit-based limiting strategy to control the OWR of tiny messages within the RNIC, thereby reducing the cache misses of these messages, such as Figure 3As shown. For small messages, the present invention does not use a credit-based restriction strategy, but instead adopts a sub-message poller. The present invention retains the cycle model of Smart and divides each cycle into an update phase and a stable phase. The cycle model assumes that the system workload changes in a cycle of 512ms. The stable phase lasts about 480ms, while the update phase lasts about 32ms. In the stable phase, a credit system is implemented for each QP that processes tiny messages. These credits represent the maximum number of WQEs that a QP can send continuously. As the number of WQEs sent by each QP increases, the credit decreases accordingly. When the credit is exhausted to zero, the QP stops sending new WQEs and polls its corresponding CQ to see if there is a CQE. After polling the CQE, the credit increases according to the number of unsignaled completions of the WQE on that QP.

[0065] Existing studies have shown that as system workload parameters (number of QPs, number of silent completions, etc.) change, the optimal concurrency depth of each QP will also change. Based on this finding, the present invention adopts a heuristic strategy to dynamically adjust the credit of each QP during the system update phase, and its algorithm is as follows (basically the same as Smart's heuristic algorithm).

[0066]

[0067] Specifically, the present invention allows the credit of each QP to fluctuate within a certain range and monitors the number of WQEs completed in a very short time (8ms). Then, the credit value that leads to the largest number of completed WQEs in this interval is selected as the basis for controlling the continuous scheduling of WQEs in the stable phase. It is worth noting that Smart has demonstrated that even if the workload variation cycle is significantly shorter than the stable phase of the cyclic model, the throttling strategy can still achieve a significant improvement in system throughput, with an improvement of more than 23.3%.

[0068] ③ (Sub-message Poller) Finally, pre-generate sub-completion queue elements (sub-CQE) for small and tiny messages to ensure that applications schedule WQEs at an appropriate rate, thereby effectively utilizing the send queue (SQ). This approach can alleviate excessive queuing delays and reduce the frequency of WQE cache misses, including the following:

[0069] The 99 tail latency for small messages remains consistently low, indicating that even a slight increase in latency can lead to a significant performance degradation for small messages. Obviously, it is inappropriate to implement a throttling policy for small messages because it introduces wait delays when credits are exhausted. A simple consideration is that no specific policy should be applied to small messages because they are insensitive to cache misses. However, when an application sets a large number of unsignaled completions, the overall message transfer rate in the system is significantly affected. While this approach may help alleviate cache misses, it severely reduces system concurrency and ultimately leads to application performance degradation.

[0070] Therefore, the present invention designs a sub-message poller to dynamically determine the number of unsignaled completions that can optimize system performance, such as Figure 4 As shown. Specifically, the present invention modifies the send_flags of WQE, allowing the WQE originally configured as unsignaled-WQE to generate CQE (sub-CQE). The purpose of this adjustment is to moderately reduce cache misses while ensuring that the transmission rate and concurrency performance of small messages are not affected. The present invention uses the wr_id field of WQE to indicate whether the WQE is designated as unsignaled-completion according to the settings of the application. When the system calls the ibv_poll_cq API to poll the sub-CQEs, these sub-CQEs will enter the middleware layer between the application and the driver, and all sub-CQEs will be discarded at this layer to ensure that the application layer is unaware of these additional CQEs.

[0071] The number of sub-CQEs generated by small messages will also change with changes in workload. Therefore, the present invention adopts a throttling strategy of a periodic model. In the update phase, according to the requirements of the application layer, the number of unsignaled completions of each QP is dynamically monitored. On this basis, the number of sub-CQEs generated with the highest throughput is determined in combination with a heuristic strategy, and this is used as the standard for the generation of sub-CQEs for small messages in the stable phase. In addition, if the number of unsignaled completions set by the application layer is too large, it may affect the replenishment of small message credits. For this reason, the present invention sets the number of sub-CQEs generated by the QP that sends small messages to one-eighth of the credit (if the credit is equal to 8, it is set to 1) based on experimental measurement results.

[0072] Embodiment 2:

[0073] Based on Example 1, but different in that WQEFC is committed to establishing middleware between the application and the user space driver, ensuring transparency between the two without changing the RNIC hardware or tightly coupling the scheduling logic with the application logic, thereby achieving flexibility and versatility. The present invention implements a prototype of WQEFC in the OFED user space library and evaluates its overhead and performance on a test platform.

[0074] In order to better meet the needs of various applications, WQEFC is implemented as a middleware between the application layer and the user space driver. WQEFC encapsulates the RDMA verbs API, enabling applications to add minimal additional parameters when calling standard APIs to utilize various components of WQEFC. By utilizing the WQEFC API, applications can dynamically adjust the concurrency depth and unsignaled completion count of WQE during communication, thereby optimizing system performance and achieving optimal throughput and latency. The prototype of WQEFC is implemented using approximately 400 lines of C++ code. WQEFC mainly provides the following APIs to the application layer:

[0075] poll_cq: This function is responsible for polling the CQ to replenish send credits, especially for tiny messages. It retrieves the number of child CQEs for these messages based on a predefined integer. It then dynamically adjusts the number of pre-generated child CQEs for small and tiny messages based on the system's concurrency depth and unsignaled completion counts.

[0076] post_send: This function distinguishes between tiny and small messages, mapping messages of different sizes to corresponding priorities. It regulates the transmission of tiny messages by managing available send credits.

[0077] During the call of the poll_cq function, a large number of child CQEs are generated. In order to maintain semantic consistency between WQEFC and the standard RDMA driver, WQEFC marks the wr_id of the WQEs that have not been signaled. When these unmarked WQEs are completed, their corresponding child CQEs are not submitted directly to the application, but are intercepted by the middleware layer and recorded in the bitmap. The role of the bitmap is to ensure that only CQEs that are not marked with wr_id can be correctly submitted to the application. Once the CQE is submitted to the application layer, the previous bitmap will be cleared and the new child CQE will be recorded. Such an implementation can ensure that the return value received by the application when calling the WQEFC API is consistent with the return value received when using the standard RDMA verbs API.

[0078] result:

[0079] The present invention uses microbenchmarks (an open source code to test RDMA performance) to evaluate three schemes: an unoptimized general method (called Common), SMART, and WQEFC. The test results show that:

[0080] (1) WQEFC utilizes the data path of mixed messages more efficiently. Compared with the state-of-the-art solution SMART, it improves the overall system throughput of mixed messages by 61.3% and maintains this high level of throughput even under high concurrency conditions.

[0081] (2) WQEFC effectively reduces the tail latency of mixed messages under high concurrency conditions. Compared with SMART, it reduces the 99th percentile tail latency of tiny messages and small messages by 41.6% and 72.6%, respectively.

[0082] The test platform used in the present invention consists of two servers and a switch. Each server is equipped with two Intel Xeon Silver 4314 CPUs (64 cores in total, hyperthreading enabled), 30GB DRAM and a 100Gbps Mellanox ConnectX-5 InfiniBand RNIC. Each RNIC is connected to the switch via a 100Gbps link. These machines are installed with Ubuntu 20.04LTS and Mellanox OpenFabrics Enterprise Distribution for Linux (MLNX_OFED, which can be understood as the driver code using the RDMA network card) v5.15.0-67-generic. In addition, each machine uses 2MB large pages to reduce the page conversion cache misses of the RNIC. In order to evaluate the impact of mixing messages of different sizes, the present invention simulates a set of data center traffic loads according to the model published by Facebook.

[0083] As mentioned above, different concurrency depths and unsignaled completion counts can significantly affect message concurrency. This paper will study the impact of different concurrency depths and unsignaled completion counts on the performance of mixed messages. Specifically, microbenchmarks are used to generate continuous traffic from the requesting machine to the receiving machine. In order to avoid QP contention within the same thread, the present invention configures 64 threads on the requesting machine, each thread is associated with a dedicated QP. In order to evaluate the impact of mixing messages of different sizes, each QP transmits 1 million messages, and the size follows the traffic load pattern published by Facebook (95% of the messages are less than 1024 bytes. Among them, 50% of the messages are less than 32 bytes). By adjusting the concurrency depth and unsignaled completion count of each QP, the present invention measures the total system throughput and flow completion time (FCT) of each message.

[0084] Throughput under concurrent testing

[0085] See also Figure 5 , Figure 5 The throughput of the three methods at different concurrency depths and unsignaled completion counts are shown, and WQEFC performs best in all scenarios.

[0086] like Figure 5As shown in (a), for mixed messages, the throughput of WQEFC is improved by 89.1\% and 26.0\% over Common and SMART, respectively. This shows that WQEFC improves data path utilization by reasonably allocating credits, thereby reducing cache misses for small messages, allowing WQEFC to maintain high throughput even at higher concurrency depths.

[0087] Figure 5 (b) shows the impact of changing the unsignaled completion count on system throughput at high concurrency depth. WQEFC achieves throughput improvements of up to 162.6% and 61.3% compared to Common and SMART, respectively. This improvement is due to WQEFC's ability to pre-generate and dynamically adjust the number of sub-CQEs, thereby reducing cache miss rates while maintaining the optimal message transmission rate and concurrency. SMART and Common lack such considerations, resulting in a drop in performance after reaching peak throughput.

[0088] Figure 5 (c) shows that in the extreme case, when the number of unsignaled completions set by the application is equal to the concurrency depth, WQEFC can maintain a high throughput. In contrast, the Common method suffers from a high cache miss rate and a decreased message transmission rate under high concurrency depth and number of unsignaled completions.

[0089] Latency under concurrent testing

[0090] See also Figure 6-7 , Figure 6 and Figure 7 The effects of the three methods on 99 tail latencies of small and tiny messages under different loads are illustrated respectively. Compared with the Common method, WQEFC improves the FCT of tiny and small messages by 49.2\% and 73.7\%, respectively. This improvement is attributed to the lack of message scheduling in the Common method, which causes all messages to be affected by cache misses. In contrast, WQEFC allocates credits to tiny messages, reducing the additional latency caused by cache misses, thereby reducing FCT. In addition, compared with the SMART method, WQEFC improves the FCT of tiny and small messages by 41.6\% and 72.6\%. Although SMART's credit-based throttling strategy reduces cache misses, it fails to optimize the performance of small messages that are less sensitive to cache misses, but instead introduces unnecessary queuing delays. WQEFC replaces credit-based throttling with pre-generated sub-CQEs, supplemented by appropriate priority scheduling, thereby reducing the queuing delay of small messages. In addition, WQEFC appropriately generates sub-CQEs for tiny messages, reducing the waiting delay associated with credit replenishment in the case of high unsignaled completion counts. Therefore, WQEFC performs well for small messages.

[0091] In addition, if Figure 6 As shown in (a), when only the concurrency depth is changed, the 99th tail delay of tiny messages of WQEFC and Smart is almost the same. Figure 6 In (b), when the unsignaled completion count is too high, Smart requires a longer credit replenishment wait time. In addition, the system may mis-evaluate the number of completed WQEs, thereby reducing the message transmission rate and increasing the 99 tail latency. In contrast, WQEFC dynamically adjusts the number of child CQEs to effectively maintain a low 99 tail latency. Therefore, when the concurrency depth and the unsignaled completion count change simultaneously, the tiny messages of WQEFC show better performance, e.g. Figure 6 (c) as shown.

[0092] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A scalable and efficient scheduling method suitable for RDMA network hybrid message scenarios, characterized in that: Includes the following: S1. Priority scheduler design: dynamically distinguish small messages from tiny messages according to the load size of the work queue elements, and assign higher priority to small messages to maximize the utilization of the data path and alleviate the additional delay of the control path caused by cache misses; S2, integral limiter design: adopt a heuristic current limiting strategy for small information to reduce the additional delay of the control path caused by frequent cache misses of small information; S3. Sub-message poller design: Pre-generate sub-completion queue elements for small and tiny messages to ensure that the application schedules work queue elements at an appropriate rate and fully utilizes the send queue to alleviate excessive queuing delays and reduce the frequency of work queue element cache misses.

2. The method for detecting coal piles by combining monocular width measurement and binocular height measurement according to claim 1 is characterized in that: The S1 specifically includes the following contents: The quality of service mechanism is used to assign higher priority to small messages to improve the utilization of the data path. Specifically: Differentiate between tiny and small messages based on the length field carried by the work queue element; map the QP corresponding to the small message to the highest priority flow controller by modifying the DSCP field in the IP header of the data packet; select the enhanced transmission selection strategy as the flow controller designated for small messages to allocate higher bandwidth; After the high-priority traffic controller has completed its service, the remaining bandwidth is allocated to the traffic controller corresponding to the small information.

3. The method for detecting coal piles by combining monocular width measurement and binocular height measurement according to claim 2 is characterized in that: The S2 specifically includes the following contents: The existing Smart algorithm is designed and adjusted to control the tiny messages in the RNIC and reduce the cache misses of the tiny messages. For small messages, a sub-message poller is used to replace the credit-based restriction strategy. The cycle model of the Smart algorithm is retained, and each cycle is divided into an update phase and a stable phase. In the stable phase, a credit system is implemented for each QP that processes tiny messages, and the credit in the credit system represents the maximum number of work queue elements sent continuously by the QP. As the number of work queue elements sent by each QP increases, the credit decreases accordingly. When the credit is exhausted to zero, the QP stops sending new work queue elements and polls its corresponding CQ to see if there are completed queue elements. After polling the completed queue elements, the credit increases according to the number of unsignaled completions of the work queue elements on the QP.

4. The method for detecting coal piles by combining monocular width measurement and binocular height measurement according to claim 3 is characterized in that: The S3 specifically includes the following contents: The sub-message poller is used to dynamically determine the number of unsignaled completions with the best system performance, and is specifically designed as follows: The send_flags of the work queue element is modified to allow the work queue element originally configured for signal-free completion to generate a completion queue element, so as to reduce cache misses without affecting the transmission rate and concurrency performance of small messages; the wr_id field of the work queue element is used to indicate whether the work queue element is designated as signal-free completion according to the application settings; when the system calls the ibv_poll_cq API to poll the child completion queue element, the child completion queue element enters the middleware layer between the application and the driver, and all child completion queue elements are discarded at this layer to ensure that the application layer is unaware of the above-mentioned additional completion queue elements.

5. A scalable and efficient scheduling system for RDMA network hybrid message scenarios applied to any of the methods described in claims 1-4, characterized in that: The system is denoted as WQEFC, and is used to establish middleware between the application and the user space driver to ensure transparency between the two. The system includes: A priority scheduler module, used to assign different priorities to small and tiny messages; The integration limiter module is used to control the concurrency depth of tiny messages; A sub-message poller module that pre-generates sub-completion queue elements for small and tiny messages.

Images