RDMA shared receiving queue optimization method, device, equipment, medium and product

Abstract

The invention relates to the technical field of remote direct memory access, in particular to an RDMA shared receiving queue optimization method and device, equipment, a medium and a product. A shared receiving queue is divided into multiple containers, and the multiple containers are connected in series through a logic linked list; when the RDMA data of the queue pair arrives, if the container is not bound, distributing a container for the queue pair; monitoring the release state of the shared receiving queue entry; and when the receiving side software monitors that the target container is in the completely idle state, re-linking the target container into the logic linked list for distribution independent of the states of other containers in the plurality of containers. According to the method disclosed by the invention, dynamic reuse of resources can be realized through resource monitoring taking the container as granularity and an out-of-order recovery mechanism independent of other container states. After all the SRQEs in any container are released, the SRQEs can be immediately put into redistribution, so that the problem of waste of tail idle resources caused by occupation of head resources in a traditional linear queue can be thoroughly avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Remote Direct Memory Access (RDMA), and in particular to an RDMA shared receive queue optimization method, device, equipment, medium and product. BACKGROUND

[0002] With the wide application of RDMA technology in high-speed data center networks, as an efficient network resource pooling scheme, the shared receive queue (SRQ) has become a core means to improve network throughput and resource utilization.

[0003] However, the SRQ management scheme in the related art faces a fundamental contradiction between resource allocation efficiency and flexibility when dealing with multi-queue pair (QP) concurrent scenarios. That is, the linear pre-allocation mode can guarantee low latency, but there is a serious head blocking problem, which causes idle resources to be unable to be reused in time; while the linked list management mode solves the resource reuse problem, but introduces additional memory read delay, which destroys the low latency characteristics of RDMA.

[0004] Therefore, how to realize efficient and low-latency allocation and recovery of SRQ resources under the premise of compatibility with multi-QP concurrency has become a key technical challenge to improve the performance of RDMA networks. SUMMARY

[0005] Therefore, the present application provides an RDMA shared receive queue optimization method, device, equipment, medium and product to at least solve the problem of how to realize efficient and low-latency allocation and recovery of SRQ resources under the premise of compatibility with multi-QP concurrency.

[0006] One aspect of the present application provides an RDMA shared receive queue optimization method, the method comprising:

[0007] The receiving side software divides the shared receive queue into multiple containers and connects the multiple containers through a logical linked list; wherein each container includes multiple shared receive queue entries;

[0008] When the receiving side network card detects the arrival of RDMA data of a queue pair, if the queue pair is not bound to a container, the receiving side network card allocates a container for the queue pair from the head of the logical linked list; the receiving side network card preferentially allocates shared receive queue entries to the arrived RDMA data in order from the container bound to the queue pair;

[0009] The receiving side software monitors the release state of the shared receive queue entries in the multiple containers in a container granularity, and determines whether the container is in a completely idle state;

[0010] When the receiving side software monitors that the target container is in a completely idle state, the target container is re-chained into the logical linked list for allocation independently of the state of other containers in the plurality of containers.

[0011] Another aspect of the present disclosure also provides an RDMA shared receive queue optimization apparatus, the apparatus comprising a container division module, a resource allocation module, a state monitoring module, and a container recycling module, wherein:

[0012] The container division module is configured to divide the shared receive queue into a plurality of containers and connect the plurality of containers through a logical linked list; wherein each container comprises a plurality of shared receive queue entries.

[0013] The resource allocation module is configured to, when detecting that the RDMA data of a queue pair arrives, if the queue pair is not bound to a container, allocate a container for the queue pair from the head of the logical linked list; and preferentially allocate shared receive queue entries to the arrived RDMA data from the container bound by the queue pair in order.

[0014] The state monitoring module is configured to monitor the release state of the shared receive queue entries in the plurality of containers in a container granularity, and determine whether the container is in a completely idle state.

[0015] The container recycling module is configured to, when monitoring that a target container is in a completely idle state, re-chain the target container into the logical linked list for allocation independently of the state of other containers in the plurality of containers.

[0016] Another aspect of the present disclosure also provides an electronic device, comprising a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the above-mentioned RDMA shared receive queue optimization method.

[0017] Another aspect of the present disclosure also provides a computer readable storage medium, the computer readable storage medium storing computer instructions, the computer instructions being used to cause a computer to implement the above-mentioned RDMA shared receive queue optimization method.

[0018] Another aspect of the present disclosure also provides a computer program product, comprising computer instructions, the computer instructions being used to cause a computer to perform the above-mentioned RDMA shared receive queue optimization method.

[0019] Through the RDMA shared receive queue optimization method, apparatus, device, medium, and product of the above-mentioned embodiments of the present disclosure, through the resource monitoring in a container granularity and the out-of-order recycling mechanism independent of the state of other containers, dynamic reuse of resources can be achieved. After all SRQEs in any container are released, the container can be immediately put into redistribution, and the problem of waste of idle resources at the tail caused by head resource occupation in the traditional linear queue can be completely avoided.

[0020] The receiving side network card is allocated in a container granularity, which can enable multiple messages of the same QP to be continuously allocated SRQEs in the container, without frequent query of the link table pointer, thereby significantly reducing the processing delay between messages of the same QP. Meanwhile, the continuous characteristic of the SRQEs in the container provides a basis for hardware prefetch optimization, thereby further reducing the memory access overhead.

[0021] In addition, the logical link table structure can maintain flexible association between containers, support concurrent access of multiple QPs, and the sequential allocation mode in the container can guarantee the locality and determinacy of operations, thereby improving resource utilization while ensuring low delay and high throughput characteristics of RDMA operations. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the specific embodiments or related art of the present disclosure, the drawings needed to be used in the specific embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0023] Figure 1 is a flowchart of a SEND operation provided by the present disclosure;

[0024] Figure 2 is a flowchart of an RDMA shared receive queue optimization method provided by an embodiment of the present disclosure;

[0025] Figure 3 is a plurality of container link diagrams of an RDMA shared receive queue optimization method provided by an embodiment of the present disclosure;

[0026] Figure 4 is an SRQ container link table structure diagram of an RDMA shared receive queue optimization method provided by an embodiment of the present disclosure;

[0027] Figure 5 is an initial sequential link structure diagram of an SRQ container of an RDMA shared receive queue optimization method provided by an embodiment of the present disclosure;

[0028] Figure 6 is a packet receiving sequence diagram of an RDMA shared receive queue optimization method provided by an embodiment of the present disclosure;

[0029] Figure 7 is a disordered sequential link structure diagram of an SRQ container of an RDMA shared receive queue optimization method provided by an embodiment of the present disclosure;

[0030] Figure 8is a structural schematic diagram of an RDMA shared receiving queue optimization device provided by an embodiment of the present disclosure.

[0031] Figure 9 is a structural schematic diagram of another RDMA shared receiving queue optimization device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] RDMA technology, as the core support of high-performance network transmission, its essence is to realize cross-device memory direct interaction by bypassing the operating system kernel, solve the problem of too high server-side data processing delay in traditional network transmission, and then meet the high-performance data interaction demand of zero-copy and low latency. SEND operation, as a typical input / output (Input / Output, IO) channel operation in RDMA technology, is the key carrier to realize cross-end data transmission, and its core process needs to rely on the cooperation of components such as QP, send queue (Send Queue, SQ), receive queue (Receive Queue, RQ) and completion queue (Completion Queue, CQ), please refer to Figure 1 , Figure 1 is a flowchart of a SEND operation provided by an embodiment of the present disclosure, which includes the following specific processes:

[0033] ① The receiving side user issues a receive queue entry (Receive Queue Entry, RQE) to the RQ in the QP;

[0034] ② The sending side user issues a request queue entry (Send Queue Entry, SQE) to the SQ in the QP;

[0035] ③ The sending side network card reads the SQE to obtain the SEND message;

[0036] ④ The sending side network card obtains the memory data;

[0037] ⑤ The sending side network card packages and sends to the opposite end, and when the data length is greater than the path maximum transmission unit (Path Maximum Transmission Unit, PMTU), it will be sent in multiple packages;

[0038] ⑥ The receiving side network card receives the message, reads the RQE to obtain the receiving buffer information; the same message uses the same RQE;

[0039] ⑦ The receiving side network card writes the data into the memory;

[0040] ⑧ The receiving side network card replies the reply message to the opposite end;

[0041] 9. The receiving side NIC reports the receiving completion information (Completion Queue Entry, CQE) to the CQ after processing the entire Message;

[0042] 10. The receiving side user obtains the receiving completion information from the CQ and recycles the RQE space;

[0043] 11. The receiving side user re-delivers the recycled RQE to the hardware for use;

[0044] 12. The sending side NIC reports the sending completion information (CQE) to the CQ;

[0045] 13. The sending side user obtains the sending completion information from the CQ and recycles the SQE space.

[0046] In the RDMA data receiving scenario, each QP is configured with an independent RQ in the conventional design, and data needs to be received in order, consumed in order, and reported to the CQE in order, but this independent RQ mode has the limitation of low resource utilization. Therefore, the SRQ technology is proposed, which allows multiple QPs to share the same SRQ, and the SEND business of multiple QPs is occupied in the order of receiving packets Shared Receive Queue Entry (SRQE): The receiving side user first issues SRQE to the SRQ, and when the receiving side NIC receives a packet pointing to a certain QP (such as QP3), the QP takes out the first SRQE from the SRQ to complete data writing, and after processing, reports CQE to the associated CQ (such as CQ3), and the user recycles the SRQE space; When a packet pointing to another QP (such as QP2) is received subsequently, the QP takes out the next SRQE to repeat the process, and the recycled SRQE is finally re-delivered to the hardware for multiplexing.

[0047] However, the conventional use mode of SRQ still has the head blocking problem. Specifically, if the receiving side first receives a SendFirst packet of a certain QP (such as QP3), the SRQE (such as SRQE0) occupied thereby needs to wait for the SendLast packet of the Message to be processed before reporting CQE and releasing; if a SendOnly packet of another QP (such as QP2) is subsequently received, the SRQE (such as SRQE1) occupied thereby can be directly reported to CQE and released in advance. But when the software delivers the SRQE in a linear (pointer incrementing) manner, because the front SRQE0 is not released, even if the rear SRQE1 is recycled in advance, it cannot be re-delivered to the hardware for use, resulting in idle resources.

[0048] To solve the head blocking problem, the related technology proposes to use SRQ linked list optimization, that is, a header is added in front of the SRQE data segment, the linked list relationship between SRQEs is maintained through next_idx in the header, the SRQEs released in advance can be concatenated into the tail of the chain, and the hardware takes SRQEs from the head of the chain each time. However, this optimization scheme still has a performance bottleneck: the linking relationship between SRQEs is random due to the difference in release timing, the hardware needs to read the header of the current SRQE from the host first to obtain the index of the next SRQE, resulting in additional host read delay between packets using the same SRQ, which seriously restricts the RDMA transmission performance and cannot meet the low latency demand of high-performance network scenarios.

[0049] In summary, the related technology has the following problems: first, the conventional linear delivery SRQE method is prone to head blocking, which causes the SRQEs released in advance to be unable to be reused in time, wasting resources; second, although the improved scheme based on the SRQ linked list solves the head blocking problem, the SRQE linking relationship is random, which introduces host read delay and seriously affects the transmission performance, and cannot adapt to the demand of high-performance RDMA data interaction.

[0050] To solve the above problems, an RDMA shared receive queue optimization method is provided in various embodiments of the present disclosure. The method comprises: a receiving side software divides a shared receive queue into multiple containers and concatenates the multiple containers through a logical linked list; wherein each container includes multiple shared receive queue entries; when the receiving side network card detects that RDMA data of a queue pair arrives, if the queue pair is not bound to a container, the receiving side network card allocates a container for the queue pair from the head of the logical linked list; the receiving side network card preferentially allocates shared receive queue entries to the arrived RDMA data in order from the container bound to the queue pair; the receiving side software monitors the release state of the shared receive queue entries in the multiple containers in a container granularity, and determines whether the container is in a completely idle state; when the receiving side software monitors that a target container is in a completely idle state, the target container is re-chained into the logical linked list for allocation independently of the state of other containers in the multiple containers.

[0051] To make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present disclosure.

[0052] Please refer to Figure 2 , Figure 2is a flowchart of an RDMA shared receiving queue optimization method provided by the embodiment of the present disclosure. The flow of the method can include the following steps:

[0053] In step S201, the receiving side software divides the shared receiving queue into multiple containers, and connects the multiple containers through a logical chain table.

[0054] In the embodiment, there are two core units in the receiving process of RDMA data, namely the receiving side software and the receiving side network card. The receiving side software is the core control unit responsible for the logical management, state monitoring and dynamic recycling and allocation of SRQ resources. Correspondingly, the receiving side network card is the core hardware unit responsible for the hardware reception, data processing and resource allocation execution of RDMA packets.

[0055] Further, the receiving side software performs structured processing on the shared receiving queue (SRQ), constructs the SRQ as an organization form with containers as basic functional units, and connects each container through a logical chain table.

[0056] Each container is configured with multiple shared receiving queue entries (SRQEs) for carrying data receiving tasks.

[0057] Here, SRQ is a receiving queue for multiple QPs to share, which realizes buffer multiplexing of multiple QP concurrent receiving tasks by uniformly managing receiving buffer resources. SRQE is a basic functional unit in SRQ, and each SRQE is associated with a host memory buffer for receiving data. The container is a basic functional unit in SRQ for carrying SRQE, and multiple SRQEs can be aggregated and managed.

[0058] In one possible implementation, the step of dividing each SRQE in the SRQ into different containers by the receiving side software can include: the receiving side software determines a preset division basis based on business requirements, wherein the preset division basis includes but is not limited to: the number of SRQEs in each container, the address range association logic of SRQEs in the memory; traversing all SRQEs in the SRQ, and logically aggregating a number of SRQEs that meet the conditions into a container according to the preset division basis.

[0059] The containers are connected through a logical chain table.

[0060] Here, the logical chain table can realize the scheduling link between containers through a preset link relationship.

[0061] As an example, refer to Figure 3 , Figure 3 is a multiple container link diagram of an RDMA shared receiving queue optimization method provided by the embodiment of the present disclosure.

[0062] As shown in Figure 3 , it is assumed that the SRQ is divided into three containers, Container0, Container1 and Container2, each of which is configured with three SRQEs. Container0 is linked to Container1 by a linked list, and Container1 is linked to Container2 by a linked list.

[0063] In addition, in one possible implementation, the number of SRQEs in a container can be flexibly configured according to actual needs.

[0064] In step S202, when the receiving side network card detects the arrival of RDMA data of a queue pair, if the queue pair is not bound to a container, the receiving side network card allocates a container for the queue pair from the head of the logical linked list; and the receiving side network card preferentially allocates a shared receiving queue entry to the arrived RDMA data in order from the container bound by the queue pair.

[0065] In this embodiment, when the RDMA data arrives at the receiving side network card, the receiving side network card needs to allocate a corresponding SRQE for the RDMA data, and accurately writes the RDMA data into the specified memory area of the receiving end host through the memory buffer parameters predefined by the SRQE.

[0066] Here, in the process of allocating the SRQE, the target container is obtained from the logical linked list of the SRQ, and at least one SRQE is selected according to a preset rule to be allocated to the RDMA data in the target container. The RDMA data can be specifically SEND data.

[0067] It can be understood that, in the process of allocating the SRQE, the receiving side network card takes a complete container as a minimum scheduling unit instead of a single SRQE, so as to avoid low scheduling efficiency caused by frequent addressing of hardware; at the same time, the container allocated by each QP corresponds to breakpoint information (such as the SRQE index allocated in the container), which is stored in the context (QPContext) of the QP, so as to ensure that the subsequent RDMA data of the same QP can be directly allocated from the unallocated SRQE position in the container in order, without the need to reposition the starting allocation point in the container.

[0068] As an example, continue to take Figure 3As shown in the three containers, it is assumed that QP1 and QP2 arrive at the receiving side network card, wherein, when the first part of RDMA data of QP1 arrives, the receiving side network card selects the first available SRQE 0 in Container 0 in sequence, and writes the first part of RDMA data into the memory buffer pre-associated with SRQE 0; when the second part of RDMA data of QP1 arrives, the receiving side network card selects the second available SRQE 1 in Container 0 in sequence, and writes the second part of RDMA data into the memory buffer pre-associated with SRQE 1; when the third part of RDMA data of QP1 arrives, the receiving side network card selects the third available SRQE 2 in Container 0 in sequence, and writes the third part of RDMA data into the memory buffer pre-associated with SRQE 2, at this time, all SRQEs in Container 0 are allocated to QP1.

[0069] When the first part of RDMA data of QP2 arrives, because QP2 is not bound to a container, the receiving side network card does not need to wait for Container 0 to be consumed, and directly obtains the next container Container 1 from the head of the logical chain table and binds it to QP2, selects the first available SRQE 3 in Container 1 in sequence, and writes the first part of RDMA data of QP2 into the memory buffer pre-associated with SRQE 3.

[0070] It can be understood that the RDMA data belonging to the same QP is preferentially allocated SRQEs in a single container in sequence, avoiding the network card frequently addressing across containers; only when all SRQEs in the current container are allocated, the next container is switched to and the breakpoint information of the QP context is updated; and multiple containers can be allocated to different QPs (for example, QP1 occupies Container 0 and QP2 occupies Container 1), each QP independently uses the container bound by itself, without waiting for other QPs to release the bound container, adapting to the scenario of multiple QP data crossing arrival.

[0071] In step S203, the receiving side software monitors the release state of the shared receiving queue entries in multiple containers in a container granularity, and determines whether the container is in a completely idle state.

[0072] In this embodiment, after the receiving side network card writes the RDMA data into the memory buffer pre-associated with the SRQE, the completion queue entry (CQE) bound to the SRQE belonging to the QP is reported to the completion queue (CQ), wherein the CQE is used to represent that the SRQE has completed data processing; the receiving side software reads the CQE from the CQ, and obtains the message that the SRQE has completed data processing, and then releases the SRQE.

[0073] Here, after reading the CQE, the receiving side software locates the SRQE according to the SRQE identifier in the CQE, updates the state of the SRQE to idle, and unbinds the SRQE from the original RDMA data.

[0074] Further, the receiving side software can continuously poll the CQs bound to the QPs and acquire the CQEs reported by the receiving side network card in real time to acquire the completion states of the SRQEs, so as to monitor the release states of the SRQEs according to the completion states of the SRQEs and determine whether the container is in the completely idle state.

[0075] As an example, continuing with the example of the three containers shown in Figure 3 If the SRQE 0 in the Container 0 has completed data transmission, and the SRQE 1 and the SRQE 2 have not completed data transmission, the receiving side software updates the state of the SRQE 0 to idle, and at this time, it is determined that the Container 0 is not in the completely idle state; if the SRQE 0 to the SRQE 2 in the Container 0 have all completed data transmission, that is, the SRQE 0 to the SRQE 2 have all been released, at this time, it is determined that the Container 0 is in the completely idle state.

[0076] In step S204, when the receiving side software monitors that the target container is in the completely idle state, the target container is re-chained into the logical linked list for allocation independently of the states of other containers in the plurality of containers.

[0077] In this embodiment, once the receiving side software monitors that a container satisfies that all the SRQEs inside the container have been released (that is, the container is in the completely idle state), the container is directly re-chained into the logical linked list of the SRQs without being restricted by the states of other containers in the SRQ, so that the container continues to be allocated by the receiving side network card.

[0078] Here, even if other containers are still in the occupied state, as long as the target container itself satisfies the completely idle state, the re-chaining process can be independently performed, and the target container does not need to wait for release in order, so that the out-of-order reuse of the container is realized, and resource idling caused by the non-release of other containers is avoided.

[0079] As an example, continuing with the example of the three containers shown in Figure 3As shown in the three containers, assuming that SRQE 0 and SRQE 1 in Container 0 are allocated to the RDMA data of QP1, SRQE 2 is idle, at this time, Container 0 belongs to a partially idle state, but is not completely idle; SRQE 3 to SRQE 5 in Container 1 are all allocated to the RDMA data of QP2, at this time, Container 1 belongs to a completely occupied state; SRQE 6 to SRQE 7 in Container 2 have all completed data processing, after the receiving side software obtains the CQE through polling the CQ, it confirms that all SRQEs in Container 2 have been released, at this time, Container 2 belongs to a completely idle state.

[0080] At this time, although Container 0 is still partially occupied, Container 1 is completely occupied and has not been released, because Container 2 itself has met the completely idle condition, the receiving side software does not need to wait for Container 0 or Container 1 to be released, and can directly re-chain Container 2 to the tail of the logical linked list of the SRQ.

[0081] Through the RDMA shared receiving queue optimization method, device, equipment, medium and product of the above embodiment of the present disclosure, through resource monitoring with a container as a granularity and an out-of-order recycling mechanism independent of the state of other containers, dynamic reuse of resources can be realized. After all SRQEs in any container are released, they can be immediately put into redistribution, which can completely avoid the problem of waste of idle resources at the tail caused by head resource occupation in the traditional linear queue. The receiving side network card allocates resources with a container as a granularity, which can enable multiple messages of the same QP to be allocated with SRQEs in a container continuously, without the need to frequently query linked list pointers, significantly reducing the processing delay between messages of the same QP. At the same time, the continuous nature of the SRQEs in the container provides a basis for hardware prefetch optimization, further reducing memory access overhead. The logical linked list structure can maintain flexible association between containers, supporting concurrent access of multiple QPs; and the sequential allocation mode in the container can ensure the locality and determinacy of operations, thereby improving resource utilization while ensuring low delay and high throughput characteristics of RDMA operations.

[0082] In one possible implementation of the above step S201, the plurality of shared receiving queue entries in the container are physically continuous in the memory.

[0083] In the present embodiment, the receiving side software divides a continuous physical address block in the host memory according to the number of SRQEs in each container, ensuring that the address block can be accessed directly by the network card through RDMA. In addition, the physical addresses corresponding to each SRQE in the physical address block are continuous, that is, the end address of a previous SRQE is equal to the start address of a next SRQE.

[0084] As an example, assume Figure 3 The address of SRQE 0 in Container 0 is A, the size of each SRQE is S, the address of SRQE 1 in Container 0 is A+S, and the address of SRQE 2 in Container 0 is A+2S. The memory size occupied by the SRQE itself is preset and fixed.

[0085] In a possible implementation, when the physical addresses of the SRQEs in the container are continuous, the receiving side network card can actively read the entire continuous physical space corresponding to the container by using the cache pre-fetching technology, and pre-load the entire container and each SRQE in the container into the cache on the card, so that the host memory read operation does not need to be frequently initiated for a single SRQE, thereby reducing the number of host access and reducing the hardware waiting delay.

[0086] By using the RDMA shared receiving queue optimization method, device, equipment, medium and product provided in the embodiments of the present disclosure, the physical addresses of the SRQEs in the container are continuous, the hardware can read an entire physical space by using the cache pre-fetching technology, and the entire container can be pre-fetched into the chip, so that the host read operation does not need to be frequently initiated for a single SRQE, thereby reducing the host interaction overhead. When the network card uses the SRQEs in the same container, the next SRQE can be quickly located without relying on the linked list index lookup, the scheduling delay between the same QP messages is reduced, and the data receiving efficiency is improved.

[0087] In a possible implementation of the above step S201, the link relationship of the logical linked list is maintained by an index pointer stored in the header information of the first shared receiving queue entry in each container.

[0088] In the present embodiment, the receiving side software writes an index pointer (referred to as next_idx) pointing to the subsequent container into the header of the first SRQE in the container when initializing the container. The next_idx can be used to indicate the physical address of the first SRQE in the next container.

[0089] In addition, the header information of the first SRQE in the container can also include the address of the first SRQE itself (referred to as cur_idx). Here, the cur_idx can be used to locate the current container when the container is recycled.

[0090] As an example, refer to Figure 4 , Figure 4 is a SRQ container linked list structure diagram of an RDMA shared receiving queue optimization method provided in the embodiments of the present disclosure.

[0091] As Figure 4As shown, the SRQ can include at least 4 containers, i.e. container 0, container 1, container 2, container 3, and at least one other container possibly concatenated subsequently. Each container can include 4 SRQEs, i.e. SRQE 0 to SRQE 3 in container 0, SRQE 4 to SRQE 7 in container 1, SRQE 8 to SRQE 11 in container 2, and SRQE 12 to SRQE 15 in container 3. The header of the first SRQE in each container includes cur_idx, next_idx, and a data segment.

[0092] The next_idx of the first SRQE (SRQE 0) in container 0 points to the first SRQE (SRQE 4) in container 1, the next_idx of SRQE 4 points to the first SRQE (SRQE 12) in container 3, the next_idx of SRQE 12 points to the first SRQE (SRQE 8) in container 2, and the next_idx of SRQE 8 is 16, which can point to a container whose first SRQE has cur_idx = 16.

[0093] Here, cur_idx = 16 can also represent a reserved extension container. When a new container is added subsequently, the cur_idx of the first SRQE of the new container can be set to 16, so that the new container can be accessed through the existing chain table without reconstructing the entire chain table.

[0094] The RDMA shared receive queue optimization method, device, equipment, medium and product of the above embodiment of the present disclosure can store the next_idx in the header of the first SRQE in the container, so that the first SRQE of the next container can be directly located through the index pointer, thereby reducing the number of host memory accesses. At the same time, the out-of-order concatenation between containers (such as container 1→container 3→container 2) is supported, which avoids scheduling blocking caused by waiting for the release of the previous container, significantly reduces the reception delay between the same QP and different QPs, and meets the high-performance transmission requirements of RDMA. The cur_idx can provide accurate positioning basis for container recycling, and the software can quickly identify the idle container through the cur_idx, so that the idle container can be re-chained to the tail of the SRQ without traversing the entire chain table, thereby avoiding head blocking problems. At the same time, only the first SRQE in the container maintains the cur_idx and the next_idx, and the non-first SRQE does not need redundant fields, which reduces the waste of memory resources and further improves the overall resource utilization of the SRQ.

[0095] In one possible implementation of the step S202, when the receiving side network card detects the arrival of RDMA data of a queue pair, if the queue pair is not bound to a container, the receiving side network card allocates a container for the queue pair from the head of the logical linked list, including:

[0096] If the queue pair is not bound to a container, a current container is obtained from the head of the logical linked list, and the current container is bound to the queue pair.

[0097] The receiving side network card preferentially allocates shared receive queue entries to the arrived RDMA data in order from the container bound to the queue pair, including:

[0098] The receiving side network card preferentially allocates shared receive queue entries to the arrived RDMA data of the queue pair in order from the current container bound to the queue pair.

[0099] When starting to allocate the current container, the index pointer in the head information of the first shared receive queue entry of the current container is read, and the next container information indicated by the index pointer is recorded in the shared receive queue context of the receiving side network card.

[0100] When all the shared receive queue entries in the current container are allocated, the next container is obtained according to the pre-stored next container information in the shared receive queue context, and the queue pair is re-bound to the next container for allocation of the subsequently arrived RDMA data of the queue pair.

[0101] In the embodiment, the receiving side network card locates the first available container (i.e., the current container) of the logical linked list by querying the current linked list head pointer recorded in the shared receive queue context (SRQC). The available container refers to a container in the logical linked list of the SRQ that is not bound to other queue pairs (QPs) and can be allocated to the target QP. The definition of the available container is not only an unoccupied container, and this determination needs to be combined with the binding state of the QP to avoid different QPs competing for the same container.

[0102] The SRQC is used to store the core control information of the SRQ, and the fields stored in the SRQC include but are not limited to: the current linked list head pointer, the next container information, the container allocation state, the QP binding identifier corresponding to each container, and the like. The QP binding identifier is used to identify whether the container is bound to the QP, so as to ensure that the receiving side network card can quickly identify the available state of the container.

[0103] For example, if the chain head recorded in the SRQC is Container 0, and Container 0 is not bound to any QP, when it is detected that RDMA data of QP1 arrives, the receiving side network card determines Container 0 as the current container, and binds Container 0 to QP1, and prepares to allocate SRQE in Container 0 for the RDMA data of QP1; subsequently, if it is detected that RDMA data of QP2 arrives, since QP2 is not bound to a container, the receiving side network card queries the next available chain head container (such as Container 1) from the SRQC, binds Container 1 to QP2, and allocates SRQE in Container 1 for QP2, thereby realizing the isolation effect of different QPs consuming SRQE in different containers.

[0104] The receiving side network card allocates SRQE for the RDMA data of the target QP in the order of the physical address of the SRQE in the bound container: the first RDMA data is allocated to the first SRQE in the bound container, for example, the first RDMA data of QP1 is allocated to SRQE 0 in Container 0 after QP1 is bound to Container 0; subsequently, the RDMA data of the QP is sequentially allocated to the next SRQE in the bound container, until all SRQEs in the bound container are allocated. That is, the same QP only consumes the SRQE in the bound container, avoiding occupying container resources across QPs.

[0105] When starting to allocate the first SRQE in the current container (which has been bound to the target QP), the receiving side network card reads the next_idx in the header of the first SRQE in the container, and writes the next container information corresponding to the next_idx into a special field in the SRQC, thereby completing preloading; the next container information can include but is not limited to the start address of the container and the address of the first SRQE, and the preloading operation is associated with the target QP, thereby ensuring that the next container is only allocated to the QP subsequently.

[0106] When all shared receive queue entries in the current container are allocated, the receiving side network card directly reads the pre-stored next container information from the SRQC, obtains the next container, and rebinds it to the current QP (rather than allocating it to other QPs), for allocation of the RDMA data of the QP subsequently; for example, after all SRQEs in Container 0 bound to QP1 are allocated, the receiving side network card rebinds the pre-stored next container (such as Container 3) in the SRQC to QP1, and the RDMA data of QP1 subsequently is directly allocated to SRQE in Container 3 in order, thereby realizing seamless switching of the same QP among containers.

[0107] By the RDMA shared receiving queue optimization method, device, equipment, medium and product of the above-mentioned embodiments of the present disclosure, by preloading the next container information to the SRQC, the receiving side network card stores the information of the subsequent container after the current container allocation process is completed, and in combination with the characteristic that the SRQE physical address in the container is continuous, the hardware cache prefetch technology can be fully utilized when allocating in order, the number of host memory accesses for single SRQE allocation is reduced, the average delay of RDMA data reception is significantly reduced, and the RDMA transmission efficiency is improved. In addition, the allocation mode with the container as the granularity, in combination with the pre-stored next container information in the SRQC, realizes the scheduling logic of seamlessly switching to the next container as soon as the current container is used up, so that even in the face of high-burst RDMA data, the network card can quickly complete the container switching through the preloaded context information, and the continuity and stability of data reception are guaranteed.

[0108] In a possible implementation of the above-mentioned embodiment, the method further includes:

[0109] The receiving side network card generates a corresponding completion queue entry after completing data processing on each shared receiving queue entry, and reports the completion queue entry to a corresponding completion queue; wherein the completion queue entry includes identification information for identifying the shared receiving queue entry;

[0110] The receiving side software monitors the release state of the shared receiving queue entries in the multiple containers in a container granularity, and determines whether the container is in a completely idle state, including:

[0111] The receiving side software receives and processes the completion queue entry by polling the completion queue;

[0112] The receiving side software determines the corresponding shared receiving queue entry and the container to which the shared receiving queue entry belongs according to the identification information in the completion queue entry;

[0113] The receiving side software updates the release state record of the container to which the shared receiving queue entry belongs, and marks that the shared receiving queue entry has been released;

[0114] The receiving side software queries the release state record of the container, and determines that the container is in a completely idle state when it is monitored that the release state of each shared receiving queue entry in the container is marked as released.

[0115] In the embodiment, the receiving side network card generates a corresponding completion queue entry after completing data processing on each shared receiving queue entry, and reports the completion queue entry to a corresponding completion queue, including:

[0116] The receiving side network card triggers the generation of the corresponding CQE immediately after completing the data processing of the single SRQE, ensures that one SRQE corresponds to one CQE, and avoids state omission; and writes identification information for identifying the SRQE in the generated CQE, wherein the identification information can uniquely identify the target SRQE and the container to which the target SRQE belongs.

[0117] The receiving side network card reports the CQE to the CQ that is previously bound to the QP according to the QP to which the SRQE belongs.

[0118] Here, each QP can uniquely correspond to one exclusive CQ to avoid confusion of state information of different QPs.

[0119] Further, the receiving side software polls the CQs bound to the QPs at a preset period, reads the CQEs to be processed in the queue; if a new CQE is detected, a subsequent processing flow is triggered immediately to avoid the delay of polling leading to the untimely release of the SRQE.

[0120] The receiving side software extracts the identification information of the SRQE in the CQE and determines the specific container corresponding to the SRQE.

[0121] In addition, the receiving side software can maintain a corresponding release state record for each container, which can include the released SRQE count, the total number of SRQEs in the container, and the like.

[0122] After locating the container, the receiving side software can add 1 to the released SRQE count in the release state record and mark the current SRQE as released to ensure the real-time synchronization of the state record.

[0123] The receiving side software can query the release state record of the target container, compare whether the released SRQE count is consistent with the total number of SRQEs in the container, if consistent, determine that the target container is in a completely idle state; if not consistent, determine that the target container is in a partially idle or completely occupied state.

[0124] Through the RDMA shared reception queue optimization method, device, equipment, medium and product of the embodiments of the present disclosure, through the strict binding mechanism that one SRQE corresponds to one CQE, the receiving side network card generates a unique CQE for each processed SRQE, and the CQE contains identification information that can uniquely identify the SRQE and the container to which the SRQE belongs, ensuring that the software can accurately associate the release state of each SRQE. By maintaining an independent release state record for each container and strictly comparing the released count and the total number to determine the completely idle state, it is ensured that only the container in which all SRQEs are released will be re-chained into the linked list, avoiding the resource conflict caused by mistakenly judging the partially idle container as available, so that the cognition of the container state by the software and hardware is completely synchronized, the risk of scheduling errors is significantly reduced, and the system stability is improved.

[0125] In one possible implementation of the step S204, re-chaining the target container into the logical linked list for allocation includes:

[0126] Re-chaining the target container into the end of the logical linked list for allocation.

[0127] In the embodiment, after the receiving side software determines that the target container is in the completely idle state, the receiving side software reads the cur_idx of the first SRQE in the target container, the total number of SRQEs in the container, and other information, and directly locates the container at the end of the current logical linked list by querying the chain tail pointer recorded in the SRQC.

[0128] The receiving side software reads the header of the first SRQE in the current end container, and updates the next_idx field of the header to the cur_idx of the first SRQE in the target container, so as to realize the linking relationship between the end container and the target container.

[0129] Further, the next_idx of the first SRQE in the target container is set to a preset chain tail identifier, so as to represent that the target container is the last node of the current linked list, and there is no other container after the target container.

[0130] The chain tail pointer recorded in the SRQC is updated to the cur_idx of the first SRQE in the target container, so that the SRQC synchronously records the latest tail position of the linked list; and the state flag of the target container is modified, so as to ensure that the receiving side network card can identify, through the SRQC, that the container is in the logical linked list and can be allocated to the newly arrived RDMA data when the receiving side network card is subsequently scheduled.

[0131] By the RDMA shared receiving queue optimization method, device, equipment, medium and product provided in the above embodiments of the present disclosure, the completely idle target container is re-chained into the end of the logical linked list, so that the container which is originally temporarily idle re-enters the scheduling link, resource waste caused by the container not being recycled is avoided, the utilization rate of the SRQE is significantly improved, and the data receiving demand of multiple QPs can be more efficiently supported in a high concurrency scenario. The logical linked list is ensured to always maintain a complete chain structure, and the receiving side network card can smoothly traverse from the head of the chain to the new tail of the chain when scheduling, without interrupting or reconstructing the linked list due to container recycling, thereby increasing the continuity of the linked list scheduling.

[0132] In one possible implementation, a specific embodiment is provided, in which, first, in the initial stage, the receiving side software serially links the containers into a logical linked list. For details, refer to Figure 5 , Figure 5 is a schematic diagram of an initial order serial linking structure of an SRQ container of an RDMA shared receiving queue optimization method provided in the embodiments of the present disclosure.

[0133] As Figure 5As shown, the SRQ includes four containers, namely container 0 to container 3, wherein each container can include four SRQEs; in the header of the first SRQE in each container, the next_idx field points to the physical address of the first SRQE of the subsequent container, for example, the next_idx of SRQE0 of container0 is equal to the address of the first SRQE (i.e. SRQE4) in container 1, container0 points to container1; container1 points to container2, and so on.

[0134] Secondly, in the normal hardware packet receiving stage, taking the packet receiving process of the QP2 and QP3 queue pair as an example, for details, please refer to Figure 6 , Figure 6 is a packet receiving sequence diagram of an RDMA shared receiving queue optimization method provided by the embodiment of the disclosure.

[0135] As Figure 6 shown, the packet receiving stage includes the following specific steps:

[0136] Step 1: processing of the first packet (Send1) of QP2.

[0137] Specifically, when the receiving side network card receives the Send1 packet of QP2, container0 is obtained from the chain head (indicated by the chain head pointer of SRQC) of the SRQ logical chain table as the current container, and container0 is bound with QP2; the first SRQE (SRQE0) in container0 is allocated for receiving data; the next_idx (pointing to SRQE4 of container1) of SRQE0 is read, and the information is recorded in SRQC to complete the preloading of the next container; at this time, the chain head pointer of SRQC is updated to SRQE4 of container1, and the original chain head container0 is marked as occupied because it has been bound with QP2.

[0138] Step 2: processing of the subsequent packets (Send2, Send3) of QP2.

[0139] Specifically, when receiving the Send2 and Send3 packets of QP2, the next SRQEs (SRQE1 and SRQE2) in container0 are directly assigned in sequence because QP2 has been bound to container0 and has not been consumed completely; no need to wait for the completion information of SRQE0 to return, nor to read the host memory to obtain the next entry index, reducing the access delay between QP packets; at the same time, the receiving side network card maintains the SRQE assignment breakpoint in container0 (such as SRQE2 has been assigned) in real time through the context of QP2, ensuring that the subsequent packets can be continuously assigned.

[0140] Step 3: Processing of the first packet (Send1) of QP3.

[0141] Specifically, when the receiving side network card receives the Send1 packet of QP3, it is detected that QP3 is not bound to a container, and there is no need to wait for the completion information of the corresponding SRQE0 of Send1 of QP2 to return (different QPs independently bind containers, resource isolation); container1 is obtained as the current container from the SRQ chain header (currently container1), and container1 is bound to QP3; the first SRQE (SRQE4) is assigned for receiving data; the next_idx of SRQE4 (pointing to SRQE8 of container2) is read and updated into the SRQC (this preloaded information is associated with QP3) for subsequent allocation of QP3.

[0142] Step 4: Processing of the subsequent packet (Send2) of QP3.

[0143] Specifically, when receiving the Send2 packet of QP3, the next SRQE (SRQE5) in container1 is directly assigned in sequence because QP3 has been bound to container1 and has not been consumed completely; no need to wait for the completion information of the previous SRQE (SRQE4), reducing the scheduling delay between different QPs.

[0144] Step 5: Switching after the container is consumed.

[0145] Specifically, when QP2 consumes all SRQEs (SRQE0-SRQE3) in container0, the receiving side network card reads the next container information (SRQE12 of container 3) associated with QP2 from the SRQC, and rebinds container 3 with QP2; when QP3 consumes all SRQEs (SRQE4-SRQE7) in container1, the network card directly reads the pre-stored next container information (such as SRQE8 of container2) from the SRQC, and rebinds container 2 with QP3; without requerying the chain head of the chain table for QP2 and QP3, the container switching is directly completed based on the associated preloaded information, realizing seamless switching between containers for the same QP.

[0146] Step 6: CQE reporting.

[0147] Specifically, after each SRQE completes data processing, the receiving side network card generates a corresponding CQE, and according to the QP (such as SRQE0 of QP2 and SRQE4 of QP3) to which the SRQE belongs, reports the CQE to the CQ bound exclusively for the QP; the CQE contains the identification information of the SRQE, which is used for subsequent accurate positioning of the binding relationship between the container and the QP by software.

[0148] Then, in the software recycling container phase, please refer to Figure 7 , Figure 7 is an SRQ container out-of-order string chain structure diagram of an RDMA shared receiving queue optimization method provided by the embodiment of the disclosure. As shown in Figure 7 , the recycling container phase includes the following specific steps:

[0149] The receiving side software polls the CQ to obtain the CQE, and locuses the corresponding SRQE and the container (such as container3) to which it belongs according to the identification information in the CQE (such as through the SRQE address to locate container3); the software updates the release state record of the container, and adds 1 to the released count for each released SRQE.

[0150] When the software detects that the released count of a certain container is equal to the total number of SRQEs (such as 4 SRQEs of container3 are all released) of the container, it is determined that the container is in a completely idle state.

[0151] The software reads the cur_idx of the first SRQE in container3; locates the end container (e.g. container1) of the current linked list through the chain tail pointer of the SRQC, updates the next_idx of the first SRQE of the end container to the cur_idx of container3; sets the next_idx of the first SRQE of container3 to NULL, and updates the chain tail pointer of the SRQC to the cur_idx of container3; marks container3 as available, so that it can be reallocated by the hardware.

[0152] When container2 completes all SRQE releases, the software repeats the steps of container reordering and chaining to the chain tail, and chains container2 to the current chain tail (after container3), to achieve the reordering and reuse of containers.

[0153] In an embodiment, an RDMA shared receive queue optimization apparatus 800 is provided, which corresponds to the RDMA shared receive queue optimization method described above. As shown in the figure, the RDMA shared receive queue optimization apparatus 800 includes a container division module 801, a resource allocation module 802, a state monitoring module 803, and a container recycling module 804, and the functions of each module are described in detail as follows: Figure 8

[0154] The container division module 801 is configured to divide the shared receive queue into multiple containers, and link the multiple containers through a logical linked list; wherein each container includes multiple shared receive queue entries.

[0155] The resource allocation module 802 is configured to, when the receiving side network card detects the arrival of RDMA data of a queue pair, if the queue pair is not bound to a container, allocate a container from the head of the logical linked list to the queue pair; and the receiving side network card preferentially allocates shared receive queue entries to the arrived RDMA data in order from the container bound to the queue pair.

[0156] The state monitoring module 803 is configured to monitor the release state of the shared receive queue entries in the multiple containers in a container granularity, and determine whether the container is in a completely idle state.

[0157] The container recycling module 804 is configured to, when a target container is detected to be in a completely idle state, independently of the states of other containers in the multiple containers, chain the target container into the logical linked list for allocation.

[0158] In an embodiment, the multiple shared receive queue entries in a container are physically continuous in memory.

[0159] ​In an embodiment, the link relationship of the logical linked list is maintained by an index pointer stored in the head information of the first shared receive queue item in each container.

[0160] In an embodiment, the resource allocation module 802 is configured to, if the queue pair is not bound to a container, obtain a current container from the head of the logical linked list, and bind the current container to the queue pair.

[0161] The shared receive queue items are allocated to the RDMA data arrived in the queue pair in order from the current container bound to the queue pair.

[0162] When the current container is started to be allocated, an index pointer in the head information of the first shared receive queue item in the current container is read, and next container information indicated by the index pointer is recorded in the shared receive queue context of the receiving side network card.

[0163] When all the shared receive queue items in the current container are allocated, a next container is obtained according to the pre-stored next container information in the shared receive queue context, and the queue pair is re-bound to the next container for allocation of the RDMA data arrived subsequently in the queue pair.

[0164] In an embodiment, the RDMA shared receive queue optimization apparatus 800 further comprises a completion queue reporting module 805, wherein:

[0165] The completion queue reporting module 805 is configured to, after the receiving side network card completes the data processing of each shared receive queue item, generate a corresponding completion queue item, and report the completion queue item to a corresponding completion queue; wherein the completion queue item comprises identification information for identifying the shared receive queue item.

[0166] The state monitoring module 803 is specifically configured to receive and process the completion queue items by polling the completion queue.

[0167] According to the identification information in the completion queue item, the corresponding shared receive queue item and the container to which the shared receive queue item belongs are determined.

[0168] The release state record of the container to which the shared receive queue item belongs is updated, and the shared receive queue item is marked as released.

[0169] The release state record of the container is queried, and when it is monitored that the release state of each shared receive queue item in the container is marked as released, the container is determined to be in a completely idle state.

[0170] In an embodiment, the container recycling module 804 is specifically configured to chain the target container to the end of the logical linked list for allocation.

[0171] It should be noted that the RDMA shared receive queue optimization device provided in the above embodiments is only illustrated by the division of the above program modules when implementing the corresponding RDMA shared receive queue optimization method. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the above system can be divided into different program modules to complete all or part of the processing described above. In addition, the system provided in the above embodiments and the corresponding Figure 2 The embodiments of the methods shown belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0172] This disclosure also provides an electronic device having the above-described features. Figure 8 The RDMA shared receive queue optimization device is shown.

[0173] Please see Figure 9 , Figure 9 This is a schematic diagram of another RDMA shared receive queue optimization device provided in this disclosure embodiment, as shown below. Figure 9 As shown, the electronic device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 9 Take a processor 10 as an example.

[0174] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0175] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0176] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required for at least one function, and the like. The data storage area can store data created according to usage of the electronic device, and the like. In addition, the memory 20 can include a volatile memory, and also include a non-volatile memory such as at least one of a magnetic disk storage device, a flash memory device, or other non-volatile solid state storage device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, which can be connected to the electronic device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0177] The memory 20 can include a volatile memory such as a random access memory, and also include a non-volatile memory such as a flash memory, a hard disk, or a solid state disk. The memory 20 can also include a combination of the above-mentioned types of memory.

[0178] The electronic device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 can be connected through a bus or other means, Figure 9 The connection through the bus is taken as an example.

[0179] The input device 30 can receive input digital or character information, and generate key signal input with respect to user settings and function controls of the electronic device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, and the like. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), a haptic feedback device (e.g., a vibration motor), and the like. The display device includes, but is not limited to, a liquid crystal display, a light emitting diode, a display, and a plasma display. In some alternative embodiments, the display device can be a touch screen.

[0180] The electronic device also includes a communication interface for communication of the electronic device with other devices or communication networks.

[0181] The embodiments of the present disclosure further provide a computer readable storage medium, and the method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or recorded in a storage medium, or be implemented as computer code originally stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded through a network and stored in a local storage medium, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk or a solid state disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that the computer, processor, microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, processor or hardware, the method shown in the above embodiments is implemented.

[0182] Part of the present disclosure can be applied as a computer program product, for example, computer program instructions, when executed by a computer, through the operation of the computer, the method and / or technical solutions according to the present disclosure can be called or provided. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source file, executable file, installation package file, etc., accordingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0183] Although the embodiments of the present disclosure are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present disclosure, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method for RDMA shared receive queue optimization, the method comprising: The method comprises: The receiving side software divides the shared receiving queue into multiple containers and connects the multiple containers through a logical linked list; each container includes multiple shared receiving queue entries; When the receiving side network card detects that RDMA data of a queue pair arrives, if the queue pair is not bound to a container, the receiving side network card allocates a container for the queue pair from the head of the logical linked list; the receiving side network card preferentially allocates shared receiving queue entries to the arrived RDMA data in the container bound to the queue pair in sequence; The receiving side software monitors the release state of the shared receiving queue entries in the multiple containers in a container granularity and determines whether the container is in a completely idle state; When the receiving side software monitors that a target container is in the completely idle state, the target container is re-chained into the logical linked list for allocation independently of the state of other containers in the multiple containers.

2. The method of claim 1, wherein, The multiple shared receiving queue entries in the container are physically continuous in memory.

3. The method of claim 2, wherein, The link relationship of the logical linked list is maintained through an index pointer in the head information of the first shared receiving queue entry stored in each container.

4. The method of claim 3, wherein, When the receiving side network card detects that RDMA data of a queue pair arrives, if the queue pair is not bound to a container, the receiving side network card allocates a container for the queue pair from the head of the logical linked list, which comprises: If the queue pair is not bound to a container, a current container is obtained from the head of the logical linked list, and the current container is bound to the queue pair; The receiving side network card preferentially allocates shared receiving queue entries to the arrived RDMA data in the container bound to the queue pair in sequence, which comprises: The receiving side network card preferentially allocates shared receiving queue entries to the arrived RDMA data in the current container bound to the queue pair in sequence; When the current container starts to be allocated, an index pointer in the head information of the first shared receiving queue entry of the current container is read, and next container information indicated by the index pointer is recorded in a shared receiving queue context of the receiving side network card; When all the shared receiving queue entries in the current container are allocated, a next container is obtained according to the pre-stored next container information in the shared receiving queue context, and the next container is bound to the queue pair for allocation of subsequent arrived RDMA data of the queue pair.

5. The method of claim 1, wherein, The method further comprises: After the receiving side network card completes data processing on each shared receiving queue entry, a corresponding completion queue entry is generated, and the completion queue entry is reported to a corresponding completion queue; the completion queue entry includes identification information for identifying the shared receiving queue entry; The receiving side software monitors the release state of the shared receiving queue entries in the multiple containers in a container granularity and determines whether the container is in a completely idle state, which comprises: The receiving side software receives and processes the completion queue entry by polling the completion queue; The receiving side software determines the corresponding shared receiving queue entry and the container to which the shared receiving queue entry belongs according to the identification information in the completion queue entry. The receiving side software updates the release state record of the container to which the shared receiving queue entry belongs, and marks that the shared receiving queue entry has been released; The receiving side software queries the release state record of the container, and determines that the container is in a completely idle state when it is monitored that the release states of all shared receiving queue entries in the container are marked as released.

6. The method of claim 1, wherein, The re-chaining of the target container into the logical chain table for allocation includes: Re-chaining the target container to the end of the logical chain table for allocation.

7. An RDMA shared receive queue optimization apparatus, comprising: The device includes a container division module, a resource allocation module, a state monitoring module, and a container recycling module, wherein: The container division module is configured to divide the shared receiving queue into multiple containers and connect the multiple containers through a logical chain table; each container includes multiple shared receiving queue entries; The resource allocation module is configured to, when detecting that RDMA data of a queue pair arrives, allocate a container from the head of the logical chain table to the queue pair if the queue pair is not bound to a container; and preferentially allocate shared receiving queue entries to the arrived RDMA data in the container bound to the queue pair in sequence. The state monitoring module is configured to monitor the release states of the shared receiving queue entries in the multiple containers in a container granularity, and determine whether a container is in a completely idle state. The container recycling module is configured to, when monitoring that a target container is in the completely idle state, re-chain the target container into the logical chain table for allocation independently of the states of other containers in the multiple containers.

8. The apparatus of claim 7, wherein, The container division module divides each container formed by the container division module, and the physical addresses of the multiple shared receiving queue entries in the container are continuous in the memory.

9. The apparatus of claim 8, wherein, The resource allocation module is specifically configured to, if the queue pair is not bound to a container, obtain a current container from the head of the logical chain table, and bind the current container to the queue pair. The resource allocation module is specifically configured to, if the queue pair is not bound to a container, obtain a current container from the head of the logical chain table, and bind the current container to the queue pair. When starting to allocate the current container, the index pointer in the head information of the first shared receiving queue entry of the current container is read, and the next container information indicated by the index pointer is recorded in the shared receiving queue context of the receiving side network card. When all shared receiving queue entries in the current container are allocated, the next container is obtained according to the pre-stored next container information in the shared receiving queue context, and is bound to the queue pair again for allocation of subsequently arrived RDMA data of the queue pair.

10. The apparatus of claim 7, wherein, The device further includes: A completion queue reporting module configured to, after the receiving side network card completes data processing of each shared receiving queue entry, generate a corresponding completion queue entry, and report the completion queue entry to a corresponding completion queue; the completion queue entry includes identification information for identifying the shared receiving queue entry. The state monitoring module is specifically configured to: Receive and process the completion queue entries by polling the completion queue. According to the identification information in the completed queue entry, a corresponding shared receiving queue entry and a container to which the shared receiving queue entry belongs are determined; A release state record of the container to which the shared receiving queue entry belongs is updated, and it is marked that the shared receiving queue entry has been released; The release state record of the container is inquired, and when it is monitored that the release states of all shared receiving queue entries in the container are marked as released, it is determined that the container is in a completely idle state.

11. The apparatus of claim 7, wherein the container recycling module is specifically configured to chain the target container to the end of the logical linked list for allocation.

12. An electronic device, comprising: Comprise: a memory for storing a computer program; a processor for implementing the steps of the RDMA shared receiving queue optimization method of any one of claims 1 to 6 when executing the computer program.

13. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and when the computer program is executed by the processor, the steps of the RDMA shared receiving queue optimization method of any one of claims 1 to 6 are implemented.

14. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor, and the steps of the RDMA shared receiving queue optimization method of any one of claims 1 to 6 are implemented.

Images