A method and system for transferring shared remote direct memory access based on reliable connection queues.

Abstract

This invention provides a method and system for remote direct memory access transmission based on shared reliable connection queues. The method includes: if the information size carried by an original work queue element (WQE) originating from the upper-layer application of the sending end is greater than a set maximum transmission unit, the original WQE is split, and the resulting sub-WQE is added to a long-flow low-priority queue; otherwise, the original WQE is added to a short-flow high-priority queue. For each scheduling period, if the short-flow high-priority queue is not empty, elements in the short-flow high-priority queue are distributed one by one to the remote direct memory access network interface card (RNIC) through a unique proxy reliable connection queue pair (RC QP) between the sending and receiving ends; otherwise, based on the window size of the current scheduling period, elements are selected from the long-flow low-priority queue according to a set transmission strategy and aggregated and distributed to the RNIC through the unique proxy RC QP. This invention can solve the head-of-queue blocking problem when sharing queue pairs.

Description

Technical Field

[0001] This invention relates to the field of computer network technology, and in particular to a method and system for remote direct memory access transmission based on a reliable connection queue. Background Technology

[0002] With the continuous improvement of data center network (DCN) transmission performance, the traditional TCP / IP protocol stack can no longer meet the needs of emerging services such as distributed storage, distributed training, and high-performance computing. Remote Direct Memory Access (RDMA) technology offloads the network protocol stack through hardware, realizing kernel bypass and zero-copy mechanisms. Due to its high throughput, low latency, and low CPU utilization transmission characteristics, it has become a core technology supporting the development of data center networks. However, as the scale of data center networks continues to increase and the number of connected nodes continues to grow, RDMA's connection scalability issues are becoming increasingly prominent: In connection-intensive scenarios such as large-scale storage and computing, services typically use a full-mesh connection (i.e., a direct private link exists between any two nodes) for communication between server nodes. A single RDMA Network Interface Card (RNIC) needs to carry hundreds of thousands of concurrent queue pairs (QP) connections. For example, in model training and inference operations, especially when using Mixed Experts (MoE) models, there are numerous all-to-all communication patterns (i.e., each node can send its stored data to other nodes), resulting in tens of thousands of concurrent connections on a single network interface card (NIC). This large number of concurrent connections puts immense pressure on the cache resources of RDMA NICs, causing frequent cache misses, a sharp decline in NIC throughput, and insufficient hardware processing power. This, in turn, leads to the spread of Priority-based Flow Control (PFC) throughout the entire data center network, causing severe network problems such as PFC storms and PFC deadlocks. Therefore, the connectivity scalability issue of RDMA has become a major bottleneck limiting the performance of large-scale data center networks.

[0003] To address the performance degradation caused by an excessive number of concurrent QP connections, the mainstream solution in academia and industry is to reduce connection size through queue-based connection sharing (i.e., packing multiple logical sessions into the same proxy QP for transmission). The connection sharing strategy essentially alleviates connection scalability issues by consolidating and reusing connection resources. This method avoids each process or communication task transmitting through a separate communication connection, improving the utilization of resources used for storing connection metadata in RDMA network cards, thereby supporting more communication requests under limited storage resource conditions.

[0004] However, connection-sharing solutions inevitably face the head-of-line blocking (HOL) problem: when multiple messages share the same queue pair, if the message at the head of the queue cannot be processed in time for some reason (such as a large message size, destination port congestion, or data verification errors), subsequent messages in the queue will be blocked after the message at the head of the queue. For latency-sensitive applications in DCN, such blocking or delay is unacceptable. Therefore, effectively solving the head-of-line blocking problem is a challenge that must be overcome when using connection-sharing strategies.

[0005] The current approach to resolving head-of-line congestion mainly involves assigning short streams (data streams with short durations and small data volumes, such as delay-sensitive messages) and long streams (data streams with longer durations and larger data volumes, such as throughput-sensitive messages) to two independent QP connections. This avoids the long stream blocking the short stream. However, this solution cannot determine the latency requirements of the data transmitted through these two QPs at the network interface card (NIC) level. Therefore, the RNIC will still mix and send the data received through these two QPs in subsequent transmissions, which may still cause congestion problems during message transmission.

[0006] Therefore, there is an urgent need for a general-purpose RDMA transmission scheme that can provide customized transmission scheduling services for data with different latency requirements, thereby solving the head-of-line blocking problem when QP connection is shared. Summary of the Invention

[0007] In view of this, embodiments of the present invention provide a method and system for remote direct memory access transmission based on a reliable connection queue for shared connections. By adopting different scheduling methods for the work requests corresponding to traffic with different latency requirements between the upper-layer application at the sending end and the local RDMA network card, the head-of-queue blocking problem during QP connection sharing can be solved, thereby improving the traffic transmission process during RDMA transmission.

[0008] One aspect of the present invention provides a method for transferring shared remote direct memory access based on a reliable connection queue, the method comprising the following steps: Receive the raw work queue element (WQE) sent by the upper-layer application at the sending end, and obtain the size of the information carried by the raw WQE; If the size of the information carried by the original WQE is greater than the set maximum transmission unit (MTU), the original WQE is divided into multiple sub-WQEs according to the MTU, and the sub-WQEs are added to the preset long-flow low-priority queue according to the division order and the transmission order; if the size of the information carried by the original WQE is not greater than the set MTU, the original WQE is added to the preset short-flow high-priority queue according to the transmission order. For each scheduling cycle, when the short-flow high-priority queue is not empty, the original WQEs in the short-flow high-priority queue are sequentially sent to the remote direct memory access network card through the unique proxy reliable connection queue pair between the sender and receiver. When the short-flow high-priority queue is empty, based on the window size of the current scheduling cycle, the sub-WQEs in the long-flow low-priority queue are selected according to the set transmission strategy, and the selected sub-WQEs are aggregated and sent to the remote direct memory access network card through the unique proxy reliable connection queue pair, thereby realizing remote direct memory access transmission between the sender and receiver; wherein, the selected sub-WQEs belong to the same original WQE.

[0009] In some embodiments of the present invention, the method further includes, for each scheduling period: Record the time when the selected sub-WQE is aggregated and sent, and obtain the generation time of the corresponding completion queue element of the selected sub-WQE from the upper layer application of the sending end; If the real-time latency calculated based on the aggregation and generation times does not meet the target latency condition, the window size of the current scheduling cycle is reduced according to the first set window threshold, and the reduced window size is used as the window size of the next scheduling cycle. If the real-time latency calculated based on the aggregation and generation times meets the target latency condition, the window size of the current scheduling cycle is increased according to the second set window threshold, and the increased window size is used as the window size of the next scheduling cycle.

[0010] In some embodiments of the present invention, the original WQE includes a field of starting address information; When the size of the information carried by the original WQE is greater than the set MTU, the method further includes: The weights of each sub-WQE obtained by segmenting the original WQE are calculated based on the size of the information carried by the original WQE and the set MTU; and The starting address of each sub-WQE is obtained by performing address offset calculation based on the starting address in the original WQE.

[0011] In some embodiments of the present invention, selecting a sub-WQE from the long-flow low-priority queue according to a set transmission strategy based on the window size of the current scheduling period includes: If there are sub-WQEs belonging to different original WQEs in the long-flow low-priority queue, a weighted round-robin method is used to select the sub-WQE from the long-flow low-priority queue based on the window size of the current scheduling period; wherein, the initial weight of the weighted round-robin is the weight calculated based on the size of the information carried by the original WQE and the MTU.

[0012] In some embodiments of the present invention, a weighted round-robin approach is used to select sub-WQEs from the long-flow low-priority queue based on the window size of the current scheduling period, including: The original WQE to be scheduled in the current scheduling period is determined according to the weight of the sub-WQE within the current scheduling period. Based on the determined segmentation order of the sub-WQEs of the original WQE, the sub-WQEs of the original WQE are selected according to the window size of the current scheduling period for aggregation and distribution; and after aggregation and distribution, the weights corresponding to the sub-WQEs of the original WQE are subtracted from the set weight threshold to obtain the weights corresponding to the sub-WQEs of the original WQE in the next scheduling period.

[0013] In some embodiments of the present invention, the method further includes, for each scheduling period: If a new original WQE is added to a short-flow high-priority queue while a sub-WQE in a long-flow low-priority queue is being scheduled, the newly added original WQE will be scheduled in the next scheduling cycle.

[0014] In some embodiments of the present invention, the selected sub-WQEs are aggregated and distributed to the remote direct memory access network interface card (DMI), including: aggregating the selected sub-WQEs in the form of a WQE linked list and distributing them to the remote direct memory access network interface card.

[0015] Another aspect of the present invention provides a remote direct memory access transmission system based on a reliable connection queue shared by the system, including a processor, a memory, and a computer program / instructions stored in the memory. The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method described in any of the above embodiments.

[0016] Another aspect of the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method described in any of the above embodiments.

[0017] Another aspect of the present invention provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any of the above embodiments.

[0018] The present invention proposes a method and system for remote direct memory access transmission based on a reliable connection queue for shared connections. By employing preemptive scheduling for elements in short-flow high-priority queues with high latency requirements and window scheduling for elements in long-flow low-priority queues with low latency requirements, it can provide customized scheduling services for the transmission of data with different latency requirements between the upper-layer application at the sending end and the RDMA network card. This solves the head-of-queue blocking problem in QP connection sharing while rationally scheduling traffic to avoid competition between different scheduling threads.

[0019] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0020] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings: Figure 1 This is a flowchart illustrating a method for transmitting shared remote direct memory access based on a reliable connection queue, according to an embodiment of the present invention.

[0022] Figure 2 This is a flowchart illustrating a method for transmitting shared remote direct memory access based on a reliable connection queue, according to another embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0024] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0025] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0026] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0027] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0028] RDMA connectivity scalability has become a major bottleneck limiting DCN performance. Existing solutions to RDMA connectivity scalability can be broadly categorized into hardware solutions, software solutions, and hardware-software co-design solutions: ① Hardware-based solutions: The core concept is to redesign the RNIC architecture to transfer RNIC hardware resources, thereby reducing RNIC resource pressure and improving connectivity scalability. Hardware-driven solutions, represented by Srnic and StaR, not only face high complexity in technology development but also significant cost pressures. More importantly, when these hardware devices are deployed in data center networks with varying architectures, they encounter significant compatibility bottlenecks, making it difficult to adapt to different infrastructure environments. Furthermore, the programmability of these innovative network cards is significantly inferior to software, resulting in insufficient flexibility and hindering continuous functional optimization and technological evolution. ② Software-based solutions: The core idea is to alleviate connectivity scalability problems and improve data transmission performance through efficient management of communication connections. A widely adopted connection management method is the connection sharing strategy. In traditional RDMA, QP (Queries Per Process) and network connection are tightly coupled. However, under a connection sharing strategy, they are decoupled, allowing multiple processes or communication tasks to share the same connection. This reduces the number of concurrent connections and alleviates connection scalability issues. For example, in a cluster of N server nodes, with P processes per node, using full-mesh communication mode, under traditional Reliable Connection (RC) transmission mode, a single network interface card (NIC) needs to maintain (N-1) × P × P connections, i.e., O(N×P). 2The number of connections is on the order of magnitude of (N-1) × P. If process-level connection sharing is adopted, only (N-1) × P connections need to be maintained, significantly reducing the number of connections. Although software-based solutions can indeed alleviate connection scalability issues to some extent through efficient connection management, the increase in software-level operations inevitably introduces additional latency and CPU overhead. In particular, factors such as "connection explosion" and "traffic load" in ultra-large clusters will further amplify latency and overhead issues, ultimately resulting in decreased throughput, soaring latency, and poor performance stability. ③ Software-hardware co-operation solutions mainly adopt a software-hardware integrated design of "hardware-level acceleration + software-level adaptation," which has advantages such as extreme performance guarantee, efficient resource reuse, and strong scalability. This solution attempts to balance performance and flexibility, but its closed implementation increases technical barriers and the possibility of widespread deployment.

[0029] In summary, solutions to connectivity scalability issues, whether designed from a software, hardware, or hardware-software co-design perspective, all have various problems and shortcomings. In contrast, while software solutions present additional latency and CPU overhead, these issues can be gradually mitigated through technical optimization. Therefore, software-based solutions are more favored by researchers than hardware solutions. Consequently, to address the performance degradation caused by high-concurrency connections in large-scale data center networks, this invention employs a software-level QP sharing strategy. Furthermore, to meet the network reliability requirements of RDMA transmission, this application selects Reliable Connection Queue Pair (RC QP) for connection sharing.

[0030] Specifically, the RC QP sharing mentioned in this application refers to the existence of only one working RC QP connection between the sender and receiver. Both long and short streams are transmitted through this single working RC QP connection; that is, data transmission between the sender and receiver is achieved through a shared RC QP. Although the upper-layer application of the sender may create multiple RC QP connections with different processes for data transmission during communication, this solution selects only one RC QP connection as the sole end-to-end proxy connection (the remaining RC QP connections are used as backup connections and not directly for actual data transmission). Only if the upper-layer application of the sender fails to acquire the corresponding Completion Queue Element (CQE) within a set communication time threshold will it select any other RC QP connection from the existing RC QP connections to replace the existing QP connection as the sole working RC QP connection between the sender and receiver (this is because if the corresponding CQE is not acquired within the set communication time threshold, it is assumed that the currently working RC QP connection has failed). This application does not impose specific limitations on the selection strategy for the unique RC QP connection between the sender and receiver; it can be selected randomly or based on the order of creation time. Furthermore, the sender and receiver in this application can be applications deployed on different service nodes.

[0031] This scheme reuses RC QP connections, ensuring that each end-to-end communication (communication from sender to receiver) retains only one proxy RC QP, thus significantly reducing the number of connections that the RDMA network card needs to maintain. However, existing schemes, in implementing RC QP-based connection scalability optimization, still cannot effectively solve the head-of-line congestion problem and rationally schedule traffic to avoid competition between different scheduling threads while effectively ensuring the low latency requirements of short-run traffic. Therefore, to address these issues, the inventors of this application, through in-depth research on the RDMA communication process and building upon existing research on connection sharing, propose an RDMA transmission optimization scheme based on RC QP sharing: by customizing the scheduling of traffic data with different latency requirements, the head-of-line congestion problem is solved while minimizing competition between different scheduling threads, thereby providing applications with a higher quality of service.

[0032] Specifically, in current RDMA transmission, the upper-layer application at the sending end submits work requests (i.e., work queue elements (WQEs)) to the local RDMA network card through a user-space driver. The local RDMA network card directly reads traffic data from the memory area registered by the upper-layer application and forwards it to the receiving end's RDMA network card via a switch. The receiving end's RDMA network card then writes the traffic data directly to the target memory address. Considering that data transmission between the upper-layer application at the sending end and the local RDMA network card is the basis for traffic scheduling, this application makes software-level improvements to the process between the upper-layer application submitting a work request and the sending end's RDMA network card reading traffic data. By adopting different scheduling schemes for work requests with different latency requirements and relying on the unique working RC QP connection between the sending and receiving ends, the problems of head-of-line blocking and reasonable subsequent traffic scheduling in the entire RDMA transmission process can be solved.

[0033] The RDMA transmission method based on RC QP sharing proposed in this application can be implemented by a software-level middleware deployed on the user-space driver layer between the upper-layer application at the transmitting end and the RDMA network card hardware. For example... Figure 1 As shown, the middleware may include a WQE segmentation module, a WQE scheduling module, and a window adjustment module. The WQE segmentation module is used to segment the large original WQE transmitted by the upper-layer application into multiple smaller sub-WQEs; the WQE scheduling module is responsible for differentiated scheduling of the original WQE and / or sub-WQEs, ensuring both the average shortest completion time of short flows and the fairness of throughput among long flows; the window adjustment module monitors the lifecycle of the original WQE and / or sub-WQEs scheduled for traffic transmission, infers the buffer pressure in the RNIC, and then adjusts the long flow scheduling window size of the WQE scheduling module to ensure that the proxy RC QP sends the original WQE and / or sub-WQEs to the local RDMA network card at a reasonable rate, thereby achieving reasonable traffic scheduling. Furthermore, since this application mainly improves the transmission process between the upper-layer application at the sending end and the local RNIC, which involves the transmission of messages with different information carrying sizes (in this application, long streams and short streams are used to distinguish messages with different information carrying sizes (mainly referring to the original WQE and / or sub-WQE)), the middleware may also include a classification module to determine whether the original WQE is a long stream or a short stream based on the information carrying size of the original WQE.

[0034] The WQE scheduler module is the key module of this solution. Its main purpose is to adopt customized scheduling strategies for work requests corresponding to different types of traffic. This not only prioritizes the latency requirements of short flows and solves head-of-line congestion, but also ensures fairness among long flows, coordinates multiple points, and provides a higher quality of service. The WQE scheduling module includes two pre-configured independent queues: a low-priority queue for long flows and a high-priority queue for short flows. These queues hold child WQEs to be scheduled and the original WQEs as queue elements, and initially (before RDMA transmission starts), both queues are empty. It should be noted that the low and high priorities mentioned in this application indicate the scheduling priority of queue elements, which is determined based on the latency requirements of the elements in the queue.

[0035] The purpose of this application is to provide customized services for different traffic types: for short flows (mostly WQEs corresponding to latency-sensitive traffic), priority is given to ensuring their latency requirements; for long flows, scheduling timing needs to be arranged reasonably to avoid competition between different scheduling threads and achieve reasonable traffic scheduling. Furthermore, this application designs a high-priority queue for short flows using original WQEs with information sizes no larger than the MTU as queue elements, while a low-priority queue for long flows uses sub-WQEs obtained by splitting original WQEs with information sizes larger than the MTU as queue elements. Therefore, the WQE scheduling module in this application can adopt different scheduling strategies for elements added to different pre-set queues: preemptive scheduling is used for original WQEs in the high-priority queue for short flows with high latency requirements, while window scheduling is used for sub-WQEs in the low-priority queue for long flows with low latency requirements.

[0036] like Figure 2 As shown, the RDMA transmission method based on RC QP sharing proposed in this application may include steps S110 to S130, wherein the WQE segmentation module executes step S120, and the WQE scheduling module executes step S130. Specifically, as follows: Step S110: Receive the raw Work Queue Elements (WQEs) sent by the upper-layer application at the sending end, and obtain the size of the information carried by the raw WQE. The upper-layer application at the sending end sends the raw WQEs to the middleware deployed in the user-space driver layer according to a specific sending order; where the specific sending order is determined by the upper-layer application (such as the CPU of the upper-layer application), it will not be elaborated here. For example, as... Figure 1 As shown, the upper-layer application at the sending end can form a WQE cache virtual queue according to a specific sending order, and send the original WQE in the WQE cache virtual queue one by one.

[0037] In addition, the original WQE is a request / command message and may include information in the following fields: the length of the traffic data to be sent, the local memory address (which may be the starting address of the local memory), the destination memory address, and user context information (such as user-defined identifiers and tags). To ensure the transmission of the information contained in the original WQE, this application also needs to obtain information related to the size of the information carried by the original WQE. For example, the original WQE may be designed to include fields related to the size of the information carried, or a corresponding request may be sent directly to the upper-layer application of the sending end to obtain the size of the corresponding information carried. This invention is not limited to these methods.

[0038] Step S120: If the size of the information carried by the original WQE is greater than the set Maximum Transmission Unit (MTU), the original WQE is divided into multiple sub-WQEs, and the sub-WQEs are added to a preset long-flow low-priority queue; if the size of the information carried by the original WQE is not greater than the set MTU, the original WQE is added to a preset short-flow high-priority queue in the order of transmission. The set MTU refers to the maximum payload allowed to pass through the shared RC QP connection path from the sender to the receiver; typically, the MTU is the same across server nodes within the same cluster.

[0039] As an example, this application considers original WQEs carrying information no larger than the set MTU as short flows and original WQEs carrying information larger than the set MTU as long flows. Therefore, during RDMA transmission, the classification module in the middleware can be used to determine whether the size of the information carried by the original WQE is greater than the set MTU: the classification module obtains the size of the information carried by the original WQE, compares the size of the message carried by the original WQE with the set MTU, and if the size of the information carried by the original WQE is greater than the set MTU, then the original WQE is determined to be a long flow; if the size of the information carried by the original WQE is no larger than the set MTU, then the original WQE is determined to be a short flow.

[0040] The main purpose of designing the WQE segmentation module is to solve the problem of excessively large WQEs being unable to transmit data. Furthermore, if the original WQE carrying more information is at the head of the queue, it will increase the queuing time for the original WQEs carrying less information. This application does not specifically design the segmentation method; it only requires that the message size of each segmented sub-WQE is less than or equal to the MTU. The number of sub-WQEs can be set according to requirements. For example, to reduce subsequent scheduling operations, this application can design the original WQE to be segmented according to a set MTU, ensuring that the message size of each segmented sub-WQE is equal to the set MTU size (the message size of the last segmented sub-WQE can be less than or equal to the set MTU size).

[0041] In some embodiments of the present invention, if the original WQE is determined to be a long flow, it is transmitted to the WQE segmentation module, segmented using a set segmentation method to obtain multiple corresponding sub-WQEs, and the sub-WQEs are transmitted to the WQE scheduling module; if the original WQE is determined to be a short flow, it is not segmented, and the original WQE is transmitted to the WQE scheduling module (the original WQE can be transmitted to the WQE scheduling module without being segmented, or it can be transmitted directly to the WQE scheduling module). Furthermore, when adding the original WQE and / or sub-WQE to the preset long-flow low-priority queue or short-flow high-priority queue of the WQE scheduling module, the scheduling process can be described separately according to the number of original WQE in the WQE cache virtual queue: If the original WQE is determined to be a short flow, it can be added to the preset short-flow high-priority queue in sequence according to the distribution order of the original WQE in the WQE cache virtual queue; if there is only one original WQE in the WQE cache virtual queue whose information size is greater than the set MTU, after splitting into multiple sub-WQEs, the sub-WQEs can be added to the preset long-flow low-priority queue in sequence according to the splitting order of the sub-WQEs; if there are multiple original WQEs in the WQE cache virtual queue whose information size is greater than the set MTU, the sub-WQEs can be added to the preset long-flow low-priority queue according to the splitting order of the sub-WQEs and the distribution order of the original WQEs corresponding to the sub-WQEs.

[0042] As an example, the original WQE may contain a field containing the starting address of local memory. If the size of the information carried by the original WQE is greater than the set MTU, each original WQE can be divided into multiple sub-WQEs pointing to contiguous memory addresses. Therefore, when multiple sub-WQEs are obtained, the starting address information corresponding to each sub-WQE can be obtained by performing address offset calculation based on the starting address contained in the original WQE, which can be expressed by the following formula: ; in, Indicates the first The starting address information corresponding to each WQE. This indicates the starting address of the native memory contained in the original WQE. This indicates the set MTU value.

[0043] It should be noted that, since WQE is continuously sent to the WQE scheduling module and continuously scheduled to the local RDMA network card during RDMA transmission, this application can divide the continuous transmission process of WQE into multiple scheduling cycles for description. In each scheduling cycle, the WQE scheduling module schedules elements in the short-flow high-priority queue and the long-flow low-priority queue respectively, and the sending method for elements in different queues is also different: each element in the short-flow high-priority queue is independent and is sent to the local RNIC individually when scheduled; however, the sub-WQEs scheduled in the long-flow low-priority queue may belong to the same original WQE, and elements in the long-flow low-priority queue can be aggregated and sent to the local RNIC when scheduled. Therefore, overall, within each scheduling cycle, elements in the short-flow high-priority queue may be scheduled multiple times, and elements (which can be one or more elements) in the long-flow low-priority queue are selected and aggregated for scheduling once according to the window size of the current scheduling cycle.

[0044] Step S130: For each scheduling period, if the short-flow high-priority queue is not empty, the WQE scheduling module will prioritize scheduling elements in the short-flow high-priority queue. Only if the short-flow high-priority queue is empty will it schedule elements in the long-flow low-priority queue based on the window size of the current scheduling period. Ultimately, all elements in the WQE scheduling module are sent to the RNIC through the unique RC QP proxy connection between the sender and receiver (if there are no elements in either the short-flow high-priority queue or the long-flow low-priority queue, the WQE scheduling module will not perform scheduling), thus achieving traffic transmission between the sender and receiver. That is, preemptive scheduling is used for the short-flow high-priority queue to ensure the latency requirements of the short flow, while window-controlled scheduling is used for the long-flow low-priority queue. After the long-flow low-priority queue has been scheduled once, the current scheduling period is considered to have ended, and the process automatically jumps to the next scheduling period.

[0045] As an example, considering the continuous transmission of WQE, there may be instances where new elements are added to the scheduling queue. Therefore, the method proposed in this application may further include: for each scheduling cycle, if a new original WQE is added to the short-flow high-priority queue while scheduling a sub-WQE in the long-flow low-priority queue, then the newly added original WQE is scheduled in the next scheduling cycle; within each scheduling cycle, if a new original WQE is added to the short-flow high-priority queue while the WQE scheduling module schedules elements, then the newly added original WQE can be scheduled in the current scheduling cycle. Furthermore, when scheduling the long-flow low-priority queue, the sub-WQE transmitted by the WQE splitting module can still be added to the queue, but the scheduling time needs to be determined based on the window size and the set transmission strategy.

[0046] In some embodiments of the present invention, when the short-flow high-priority queue is not empty, the original WQEs in the short-flow high-priority queue are sequentially sent to the remote direct memory access network card according to the order in which the elements in the short-flow high-priority queue are sent, until the short-flow high-priority queue is empty; when the short-flow high-priority queue is empty, a specific number of sub-WQEs are selected from the long-flow low-priority queue based on the window size of the current scheduling period, and the selected elements are aggregated and sent to the remote direct memory access network card. Specifically, this application designs that the sub-WQEs selected in each scheduling period belong to the same original WQE. Therefore, the number of the specific number of sub-WQEs selected must not exceed the window size of the current scheduling period (if the number of sub-WQEs corresponding to a certain original WQE in the long-flow low-priority queue is greater than the window size of the current scheduling period, then the specific number is equal to the window size of the current scheduling period; otherwise, the specific number is less than the window size of the current scheduling period).

[0047] More specifically, the preemptive scheduling in this application refers to "preemptive scheduling" in a broad sense, mainly including the following two principles: ① When the short-flow high-priority queue is not empty, elements in this queue are scheduled first, while long-flow low-priority queues are queued; ② If the short-flow high-priority queue was originally empty, and the WQE scheduling module is scheduling elements in the long-flow low-priority queue, if a new element is added to the short-flow high-priority queue, the WQE scheduling module, after scheduling the sub-WQEs within the window it is currently processing, immediately concentrates resources to process the new element in the short-flow high-priority queue. The window-controlled scheduling in this application means that when the short-flow high-priority queue is empty, and the WQE scheduling module is scheduling elements in the long-flow low-priority queue, only long-flow segments (a number of sub-WQEs) within the window are scheduled in each scheduling cycle, while long-flow segments outside the window need to wait for the next scheduling cycle.

[0048] Furthermore, when elements selected from the long-flow low-priority queue are aggregated and distributed within each scheduling cycle, the WQE scheduling module (or middleware) will actively trigger the RNIC's doorbell mechanism once. With the doorbell mechanism's frequency remaining constant, the number of sub-WQEs corresponding to the same number of doorbell triggers will increase. However, the RDMA network card's WQE cache resources are limited. Increasing the number of WQEs distributed without restriction may lead to a risk of WQE cache overflow. Therefore, this application designs a window size for scheduling elements in the long-flow low-priority queue in each scheduling cycle, which is updated through a window adjustment module. The specific process is as follows: Step S01: Before the end of the current scheduling cycle or the start of the next scheduling cycle, the window adjustment module determines the time for the aggregation and distribution of elements selected from the long-flow low-priority queue, and obtains the generation time of the CQE corresponding to the aggregated and distributed elements from the upper-layer application at the sending end; wherein, the time for the aggregation and distribution of elements is the time of the scheduling doorbell mechanism. The sending upper-layer application can obtain the CQE (Completion Quality Equation) through polling or interruption (the receiving end sends an ACK to the sending end's network card, which generates a corresponding CQE and places it in the completion queue), thereby knowing the task's completion status. Since the CQE can contain field information about the time of its generation, the window adjustment module can continuously send corresponding requests to the sending upper-layer application, or the sending upper-layer application can directly issue requests, to obtain the generation time of the CQE corresponding to the aggregated and issued elements in the long-flow low-priority queue (the corresponding CQE can be determined by the identification information of the aggregated and issued elements). .

[0049] Step S02: If the real-time latency calculated based on the aggregation and generation times does not meet the target latency condition, then the window size of the current scheduling period is reduced according to the first set window threshold, and the reduced window size is used as the window size of the next scheduling period; otherwise, the window size of the current scheduling period is increased according to the second set window threshold, and the increased window size is used as the window size of the next scheduling period.

[0050] In this application, the target latency condition can be set such that the difference between the aggregated delivery time and the generation time is not greater than the set target latency threshold. This invention does not impose specific limitations on this. For example, the target latency condition can also be that the ratio calculated based on the aggregation delivery time and the generation time is not greater than a set proportional threshold. Reducing the window size of the current scheduling period can be achieved by using the difference between the current scheduling period's window size and a first set window threshold as the window size of the next scheduling period. Increasing the window size of the current scheduling period can be achieved by using the sum of the current scheduling period's window size and a second set window threshold as the window size of the next scheduling period. The methods described above for calculating the window size of the next scheduling period are merely examples; the window size of the current scheduling period can also be adjusted through proportional scaling or other methods.

[0051] As an example, suppose the RDMA network card can carry the CQE generation time in the CQE, and the window adjustment module calls the first... ( Within (positive integer) scheduling cycles, the number of elements in the long-running low-priority queue that are called... and The calculation process for adjusting the window size of the current scheduling cycle based on these two timestamps can be as follows: A target latency threshold can be preset. By comparing the calculated real-time delay and target latency threshold This is used to adjust the window size of the current scheduling cycle, thereby controlling the distribution rate of sub-WQEs. ; , ; , ; in, Indicates the first The window size for each scheduling cycle, Set a threshold for the first window. Set a window threshold for the second one. , This is a hyperparameter and will be adjusted for different networks. Additionally, the window size for the first scheduling cycle is a custom value.

[0052] For users, the specific real-time WQE cache pressure of the network card is unknown. Therefore, this application designs an indirect way to infer the WQE cache pressure of the network card by observing changes in the WQE lifecycle. This allows for reasonable adjustment of the window size during long-duration window-controlled scheduling, preventing both excessive pressure on the RNIC cache and network card starvation. For example, the simple additive increase-additive decrease (AIAD) strategy can be used to adjust the window size in different scheduling cycles, thereby dynamically achieving a relatively stable WQE delivery rate.

[0053] Furthermore, when transmitting elements in the long-flow low-priority queue, the transmission strategy in this application can be set independently. For example, the sub-WQEs in the long-flow low-priority queue can be transmitted sequentially according to the segmentation order of the sub-WQEs and the distribution order of their original WQEs. Alternatively, this application can also design the elements in the long-flow low-priority queue to use a weighted round-robin (WRR) method to sequentially poll the sub-WQEs corresponding to different original WQEs to ensure the fairness of the long flow. The following will specifically explain the process of transmitting elements in the long-flow low-priority queue when the transmission strategy is set to weighted round-robin.

[0054] In some embodiments of the present invention, selecting sub-WQEs from the long-flow low-priority queue according to a set transmission strategy based on the window size of the current scheduling period includes: if there are sub-WQEs belonging to different original WQEs in the long-flow low-priority queue, a weighted round-robin method is used to select a specific number of sub-WQEs from the long-flow low-priority queue based on the window size of the current scheduling period. Furthermore, if, during or before the weighted round-robin process, there are only sub-WQEs belonging to the same original WQE in the long-flow low-priority queue, a specific number of sub-WQEs are selected and aggregated for delivery according to the segmentation order.

[0055] Specifically, a weighted round-robin approach is used to select sub-WQEs from the long-flow low-priority queue based on the window size of the current scheduling period. This includes: determining the original WQE to be scheduled in the current scheduling period according to the weights of the sub-WQEs within the current scheduling period; selecting the sub-WQEs of the original WQE based on the segmentation order of the determined sub-WQEs and the window size of the current scheduling period for aggregation and distribution; and after aggregation and distribution, subtracting a set weight threshold from the weights of the sub-WQEs of the original WQE (the weights of the sub-WQEs of other unscheduled original WQEs remain unchanged) to obtain the weights of the sub-WQEs of the original WQE in the next scheduling period. The value of the set weight threshold can be set by the user. Additionally, if the weights of all sub-WQEs of the original WQE are 0, the initial weights can be used to reset the weights before weighted round-robin selection of elements in the long-flow low-priority queue.

[0056] As an example, the initial weights for weighted round-robin are calculated based on the size of the information carried by the original WQE and the MTU. To facilitate the WQE scheduling module's weighted round-robin polling of sub-WQEs in long-flow low-priority queues and ensure fairness among long flows, this application can also calculate the weights corresponding to each sub-WQE obtained from the original WQE based on the size of the information carried by the original WQE and the set MTU, and then split the original WQE after the weight calculation. The weight calculation formula can be expressed as: ; in, Indicates the first The weights corresponding to each WQE This indicates the size of the information carried by the original WQE. According to this formula, the weights of each sub-WQE obtained from the same original WQE are the same.

[0057] For example, suppose there are three different original WQEs in the long-flow low-priority queue (denoted as A1, A2, and A3 respectively), and the weights of the sub-WQEs corresponding to A1, A2, and A3 are W1=3, W2=2, and W3=1 respectively (sub-WQEs of the same original WQE have the same weight). Setting the weight threshold to 1, the specific process of scheduling elements in the long-flow low-priority queue using a weighted round-robin method can be as follows: In the first scheduling cycle, the original WQE to be scheduled is determined based on the weight, and that is the original WQE corresponding to W1. In the first scheduling cycle, a specific number of sub-WQEs corresponding to A1 are selected and aggregated according to the segmentation order and the window size of the scheduling period. In the second scheduling cycle, since the weights of A1, A2, and A3 are 2, 2, and 1 respectively, the original WQEs to be scheduled can be selected from A1 and A2 according to the set weighted selection strategy. Assuming A2 is selected as the original WQE to be scheduled, a specific number of sub-WQEs corresponding to A2 can be selected and aggregated according to the segmentation order and the window size of the second scheduling cycle. In the third scheduling cycle, since the weights of A1, A2, and A3 are 2, 1, and 1 respectively, assuming the weighted selection strategy is set, the original WQEs to be scheduled can be selected from A1 and A2. A1 is selected from A1 and A3 as the original WQE to be scheduled. A specific number of sub-WQEs corresponding to A1 can be selected and aggregated and distributed according to the splitting order and the window size of the third scheduling cycle. In the subsequent fourth, fifth, and sixth scheduling cycles, A1, A2, and A3 can be selected as the original WQEs to be scheduled according to the weighted selection strategy. Therefore, after the end of the sixth scheduling cycle, the weights corresponding to the sub-WQEs of A1, A2, and A3 are all 0. Therefore, the weights corresponding to the sub-WQEs of A1, A2, and A3 can be reset to 3, 2, and 1 respectively, and scheduling continues based on the reset weights in the seventh scheduling cycle until all elements are scheduled. The weighted selection strategy is the strategy to select the original WQE to be scheduled when the weights of the sub-WQEs belonging to different original WQEs are the largest and the same. This strategy can be designed by the user. For example, the weighted selection strategy can be to randomly select an original WQE from the original WQEs with the largest corresponding weight as the original WQE to be scheduled.

[0058] As an example, this application can also design a WQE linked list to aggregate and distribute the sub-WQEs within each periodic window. That is, the sub-WQEs within the entire window only trigger the doorbell mechanism call once, thereby reducing the number of doorbell mechanism calls for distributing WQEs, reducing communication latency and increasing throughput.

[0059] The method proposed in this application aims to provide better service quality for connection-intensive services in data centers, and has the following significant advantages: ① Based on RC QP sharing, it can effectively alleviate the performance degradation problem caused by excessive concurrent connections during RDMA communication; ② It can provide customized scheduling strategies for the transmission of different types of messages, and adopts message segmentation and priority scheduling strategies to prioritize the latency requirements of short flows while taking into account the fairness among long flows, effectively solving the HOL problem in connection sharing strategies; ③ It also considers the WQE cache pressure of the network card, and alleviates the WQE cache loss problem caused by unlimited WQE distribution by adjusting the long flow window size; ④ Sub-WQEs are distributed through linked list aggregation and doorbell is executed only once, which can effectively reduce communication latency and improve throughput.

[0060] Corresponding to the above method, the present invention also provides a remote direct memory access transmission system based on a reliable connection queue for sharing. The system includes a computer device, which includes a processor and a memory. The memory stores computer programs / instructions, and the processor is used to execute the computer programs / instructions stored in the memory. When the computer programs / instructions are executed by the processor, the system implements the steps of the method described above.

[0061] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0062] This invention also provides a computer program product storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer program product can be a tangible product, such as random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of product known in the art.

[0063] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0064] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0065] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for transmitting shared remote direct memory access based on a reliable connection queue, characterized in that, The method includes the following steps: Receive the raw work queue element (WQE) sent by the upper-layer application at the sending end, and obtain the size of the information carried by the raw WQE; If the size of the information carried by the original WQE is greater than the set maximum transmission unit (MTU), the original WQE is divided into multiple sub-WQEs according to the MTU, and the sub-WQEs are added to a preset long-flow low-priority queue according to the division order and the transmission order; if the size of the information carried by the original WQE is not greater than the set MTU, the original WQE is added to a preset short-flow high-priority queue according to the transmission order. For each scheduling period, when the short-flow high-priority queue is not empty, the original WQEs in the short-flow high-priority queue are sequentially sent to the remote direct memory access network card through the unique proxy reliable connection queue pair between the sender and receiver. When the short-flow high-priority queue is empty, based on the window size of the current scheduling period, the sub-WQEs in the long-flow low-priority queue are selected according to the set transmission strategy, and the selected sub-WQEs are aggregated and sent to the remote direct memory access network card through the unique proxy reliable connection queue pair, thereby realizing remote direct memory access transmission between the sender and receiver; wherein, the selected sub-WQEs belong to the same original WQE.

2. The method according to claim 1, characterized in that, For each scheduling cycle, the method further includes: Record the time when the selected sub-WQE is aggregated and sent, and obtain the generation time of the completion queue element corresponding to the selected sub-WQE from the upper layer application of the sending end; If the real-time latency calculated based on the aggregation and generation times does not meet the target latency condition, the window size of the current scheduling cycle is reduced according to the first set window threshold, and the reduced window size is used as the window size of the next scheduling cycle. If the real-time latency calculated based on the aggregation and generation times meets the target latency condition, the window size of the current scheduling cycle is increased according to the second set window threshold, and the increased window size is used as the window size of the next scheduling cycle.

3. The method according to claim 1, characterized in that, The original WQE contains a field containing information about the starting address; When the size of the information carried by the original WQE is greater than the set MTU, the method further includes: The weights corresponding to each sub-WQE obtained by dividing the original WQE are calculated based on the size of the information carried by the original WQE and the set MTU. as well as The starting address of each sub-WQE is obtained by performing address offset calculation based on the starting address in the original WQE.

4. The method according to claim 3, characterized in that, The step of selecting a sub-WQE from the long-flow low-priority queue according to a set transmission strategy based on the window size of the current scheduling period includes: If there are sub-WQEs belonging to different original WQEs in the long-flow low-priority queue, a weighted round-robin method is used to select the sub-WQE from the long-flow low-priority queue based on the window size of the current scheduling period; wherein, the initial weight of the weighted round-robin is the weight calculated based on the size of the information carried by the original WQE and the MTU.

5. The method according to claim 4, characterized in that, The step of selecting a sub-WQE from the long-flow low-priority queue using a weighted round-robin method based on the window size of the current scheduling period includes: The original WQE to be scheduled in the current scheduling period is determined according to the weight of the sub-WQE within the current scheduling period. Based on the determined segmentation order of the sub-WQEs of the original WQE, the sub-WQEs of the original WQE are selected according to the window size of the current scheduling period for aggregation and distribution; and after aggregation and distribution, the weights corresponding to the sub-WQEs of the original WQE are subtracted from the set weight threshold to obtain the weights corresponding to the sub-WQEs of the original WQE in the next scheduling period.

6. The method according to claim 1, characterized in that, For each scheduling cycle, the method further includes: If a new original WQE is added to a short-flow high-priority queue while a sub-WQE in a long-flow low-priority queue is being scheduled, the newly added original WQE will be scheduled in the next scheduling cycle.

7. The method according to claim 4, characterized in that, The step of aggregating and distributing the selected sub-WQEs to the remote direct memory access network interface card includes: aggregating and distributing the selected sub-WQEs to the remote direct memory access network interface card in the form of a WQE linked list.

8. A remote direct memory access transmission system based on a reliable connection queue for shared resources, comprising a processor, a memory, and computer programs / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images