Memory access control system for RDMA network cards

Abstract

The present invention discloses a memory access control system for an RDMA network card, comprising: a memory access control software module and a memory access control logic module; the memory access control software module is arranged in a host RDMA network card driver, and is used to register and deregister memory areas and memory windows, and to map virtual address spaces to physical address spaces; the memory access control logic module is arranged in the RDMA network card, and is used to determine whether the access rights requested by the RDMA are legal, and when the access rights are legal, the virtual address is converted to the physical address, and information is provided for the DMA operation of the RDMA network card. The present invention is beneficial in that the memory access control system for the RDMA network card provided unloads the query logic related to the virtual-physical address conversion and the memory access right control to the RDMA network card hardware, which not only helps to give full play to the performance of the high-speed transmission of the network card, improves the transmission bandwidth of the network card, reduces the delay of the RDMA read and write operations, but also reduces the storage space occupation.

Description

Technical Field

[0001] The invention belongs to the field of network technology, and in particular relates to a memory access control system for an RDMA network card. Background Art

[0002] RDMA (Remote Direct Memory Access) networks have become the mainstream of data center networks in recent years due to their excellent characteristics of low latency, low CPU overhead and high bandwidth. RDMA networks include Infiniband networks and RoCE networks. RDMA supports data transfer directly from the memory of one computer to the memory of another computer without the intervention of the operating systems of both parties. RDMA technology eliminates the overhead of copying and context switching of data packets in user space and kernel space, thereby freeing up memory bandwidth and CPU cycles to improve application system performance. Most RDMA networks use dedicated smart network cards to offload network services to hardware, further reducing the CPU load.

[0003] RDMA provides a set of software transmission interfaces, through which applications can create transmission requests (WorkRequest, referred to as WR). WR describes the content that you want to interact with the other end, such as memory address, length, and key. The memory address in WR is a virtual address. The virtual memory must have been registered, have remote access permissions, and the key must be correct to allow the other end's RDMA request. Taking the RDMA write operation (RDMA Write) request as an example, the initiator puts the memory start address, write length, and key to be written by the other end into WR; the responding end network card parses the WR, and needs to verify that the memory has been registered, has remote write permissions, and the key is correct, and then converts the virtual address to the physical address. The network card uses the physical address to initiate DMA access to the host memory.

[0004] Some RDMA network cards obtain physical addresses and memory access rights by transmitting memory read and write requests and virtual addresses to the driver. This method has good flexibility, but because it involves interaction with the driver, the software overhead is large and the latency is high, so it is not suitable for RDMA networks with high bandwidth and latency requirements. Some RDMA network cards save a copy of the host address mapping table and use the CAM (Content Address Memory) resources in the chip for storage and address retrieval. However, this method requires constant updating of the cache in the network card. Due to the limited CAM resources in the chip, there may be a large number of query misses, resulting in reduced query efficiency. Therefore, in order to improve the query efficiency of virtual-physical address conversion, reduce the storage resource usage in the network card, enhance the parallelism of memory access control and address conversion, and improve the versatility of the system, further research on memory access control technology solutions is needed. Summary of the invention

[0005] The present invention provides a memory access control system for an RDMA network card to solve the above-mentioned technical problems, and specifically adopts the following technical solutions:

[0006] A memory access control system for an RDMA network card, comprising a memory access control software module and a memory access control logic module;

[0007] The memory access control software module is set in the host RDMA network card driver, and is used to register and deregister the memory area and the memory window, and to map the virtual address space to the physical address space;

[0008] The memory access control logic module is set in the RDMA network card. The memory access control logic module is used to determine whether the access rights of the RDMA request are legal, and if the access rights are legal, convert the virtual address to the physical address to provide information for the DMA operation of the network card.

[0009] Further, the memory access control software module manages a memory registration list for storing registered memory areas and memory window related information;

[0010] The memory access control software module also manages a B+ tree data structure for storing a mapping of virtual addresses to physical addresses;

[0011] The memory access control logic module uses the storage resources in the RDMA network card hardware to save the copy of the memory registration list and the B+ tree data structure. The memory access control software module actively initiates the update of the memory registration list and the B+ tree data structure in the RDMA network card hardware to ensure the consistency of software and hardware storage;

[0012] The memory access control logic module determines whether the access rights of the RDMA request are legal by accessing the memory registration list, and obtains the mapping from the virtual address to the physical address by querying the B+ tree data structure.

[0013] Furthermore, the memory access control software module includes:

[0014] A virtual-physical address mapping unit is used to map the virtual address requested by the application program to a unified IO virtual address space, and then perform virtual-physical address conversion to map a continuous virtual address space into one or more continuous physical address segments;

[0015] A B+ tree management unit, used to use the registered virtual address as the key of the B+ tree, and the registered memory length and physical segmentation information as the value of the B+ tree, and save them into the B+ tree data structure;

[0016] The memory area / memory window management unit is used to perform registration and deregistration of memory areas and memory windows, generate access keys including local access keys and remote access keys, and store information such as valid identifiers, access keys, registration addresses and lengths of virtual address spaces, address types, and access rights in a memory registration list.

[0017] Furthermore, the memory access control software module stores the memory registration list in the system memory in the form of an array;

[0018] The elements of the array of the memory registration list contain: effective identification, local access key, remote access key, registration address and length of the virtual address space, address type and access rights.

[0019] Furthermore, the valid flag indicates whether the array element is valid. When the memory area / memory window is successfully registered, the valid flag is 1, and when the memory area / memory window is unregistered, the valid flag is 0;

[0020] The local access key and remote access key are valid when the memory area / memory window has local access permission and remote access permission, respectively. Both are 32-bit integers, of which the upper 24 bits are the query index of the memory registration list and the lower 8 bits are the globally unique key.

[0021] The registered address and length of the virtual address space are the starting address and total registered length of the virtual address space specified when registering the memory region or memory window;

[0022] The address types include ordinary addresses and zero-based addresses, which are represented by the numbers 0 and 1 respectively;

[0023] Access permissions are divided into local write, remote read, remote write and remote atomic operation, and are represented by a 4-bit mask.

[0024] Further, the memory access control software module stores the B+ tree data structure in the system memory in the form of a tree structure node array and a content node array;

[0025] The tree structure node array stores the middle nodes and leaf nodes of the B+ tree. The middle node elements contain the key and the pointers to the left and right subtree nodes. The leaf node elements contain the key and the pointer to the content array elements. The content node array stores the complete key-value pairs. The content node elements correspond to the leaf node elements. Each content node element stores the pointer to its right sibling node element, which is convenient for range query.

[0026] The maximum number of elements contained in each B+ tree node is determined when the network card is initialized;

[0027] The maximum number of physical segments contained in each B+ tree element is determined when the network card is initialized;

[0028] In a memory mapping process, if the number of physical segments after the virtual address space is mapped exceeds the maximum value, the virtual address and physical segment information are split into multiple B+ tree elements for storage;

[0029] The creation, update and deletion of the B+ tree can only be processed by the memory access control software module, and the processing results are synchronized to the memory access control logic module, so that the B+ tree virtual-physical address conversion table in the RDMA network card is consistent with the B+ tree data structure content in the system memory.

[0030] Further, the memory access control logic module includes a memory access permission matching logic unit, a virtual-physical address translation logic unit, a memory registration list storage unit and a B+ tree storage unit;

[0031] The memory access permission matching logic unit takes the virtual address, length and key in the RDMA request as input, takes the high 24 bits of the key as the index of the memory registration list, obtains the memory registration information, and then matches the RDMA request with the information in the memory registration table. When the memory area in the RDMA request is within the memory registration range, the request key is completely equal to the registered key, and the requested read and write permissions are within the registered permission range, the memory access permission matching is successful, otherwise the permission matching fails;

[0032] The virtual-physical address conversion logic unit uses the virtual address in the RDMA request as a key to query the B+ tree virtual-physical address conversion table. The B+ tree query is a range query. If the virtual address is found to be within the memory range represented by the length information in the key and value of a B+ tree element, the query is successful.

[0033] The memory registration list storage unit is used to store a copy of the memory registration list;

[0034] The B+ tree storage unit is used to store a copy of the B+ tree data structure.

[0035] Furthermore, the memory access control system for the RDMA network card is applicable to an RDMA network card device based on an Infiniband network or a RoCE network.

[0036] The present invention is also beneficial in that the memory access control system provided for the RDMA network card offloads the query logic related to virtual-physical address conversion and memory access permission control to the RDMA network card hardware, which not only helps to give full play to the high-speed transmission performance of the network card, improves the transmission bandwidth of the network card, reduces the latency of RDMA read and write operations, but also reduces the storage space occupied.

[0037] The present invention is also beneficial in that it provides a memory access control system for an RDMA network card, proposes storing a virtual-physical address conversion table based on a B+ tree data structure, facilitates rapid updating and querying, and can quickly convert virtual addresses to physical addresses when processing RDMA requests that are transmitted starting from any location in the registered memory space.

[0038] The present invention is also beneficial in that the memory access control system for the RDMA network card provided uses the network card driver software to manage the B+ tree virtual-physical address translation table and the memory registration list, and stores copies of the B+ tree and the memory registration table in the RDMA network card hardware, only allowing the RDMA network card hardware to perform query operations, thereby simplifying the logical design of the network card hardware and reducing query latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 A schematic diagram of a memory access control system for an RDMA network card of the present invention;

[0041] Figure 2 A schematic diagram of the internal structure of the memory access control software module of the present invention;

[0042] Figure 3a A schematic diagram of a B+ tree storing key-value pairs provided for an exemplary embodiment of the present invention;

[0043] Figure 3b A schematic diagram of a B+ tree storing key-value pairs provided for another exemplary embodiment of the present invention;

[0044] Figure 3c A schematic diagram of a B+ tree data structure provided for an exemplary embodiment of the present invention;

[0045] Figure 4 A schematic diagram of a memory registration list array provided for an exemplary embodiment of the present invention;

[0046] Figure 5 It is a schematic diagram of the internal structure of the memory access control logic module of the present invention. DETAILED DESCRIPTION

[0047] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0048] like Figure 1 The figure shows a memory access control system for an RDMA network card of the present application, which includes a memory access control software module 10 and a memory access control logic module 20 .

[0049] The memory access control software module 10 is set in the host RDMA network card driver, and is used to register and deregister the memory area and the memory window, and to map the virtual address space to the physical address space.

[0050] The memory access control logic module 20 is set in the RDMA network card. The RDMA network card is based on ASIC or FPGA. The memory access control logic module 20 is used to determine whether the access rights of the RDMA request are legal, and if the access rights are legal, convert the virtual address to the physical address to provide information for the DMA operation of the network card.

[0051] In an embodiment of the present application, the memory access control software module 10 manages a memory registration list for storing registered memory areas and memory window related information.

[0052] The memory access control software module 10 also manages a B+ tree data structure for storing the mapping of virtual addresses to physical addresses.

[0053] The memory access control logic module 20 uses the storage resources in the RDMA network card hardware to save copies of the memory registration list and the B+ tree data structure. The memory access control software module 10 actively initiates updates to the memory registration list and the B+ tree data structure in the RDMA network card hardware to ensure the consistency of software and hardware storage. The memory registration list and the B+ tree data structure managed by the memory access control software module 10 have the same storage format and content as the memory registration list and the B+ tree data structure managed by the memory access control logic module 20. The memory access control logic module 20 determines whether the access rights requested by the RDMA are legal by accessing the memory registration list, and obtains the mapping of the virtual address to the physical address by querying the B+ tree data structure.

[0054] In the implementation manner of the present application, the memory access control software module 10 includes: a virtual-physical address mapping unit 11 , a B+ tree management unit 12 , and a memory area / memory window management unit 13 .

[0055] Among them, the virtual-physical address mapping unit 11 is used to map the virtual address requested by the application program to the unified IO virtual address space, and then perform virtual-physical address conversion, mapping a continuous virtual address space into one or more continuous physical address segments. Specifically, the starting virtual address and memory length requested by the application program are denoted as <UVA, L>, and the starting virtual address and length mapped to the unified IO virtual address space are denoted as <IOVA, L>. After address conversion, a continuous virtual address space will be mapped into N (N is a positive integer) continuous physical address segments. The i-th physical segment is denoted as <PAi, Offseti>, and all physical segments are denoted as {<PAi, Offseti>|0 <= i < N}, where PAi is the starting physical address of the i-th segment, and Offseti is the offset of the starting position of the i-th segment in the entire memory registration area.

[0056] The B+ tree management unit 12 is used to, in the case of receiving a memory registration request, take the registered virtual address as the key of the B+ tree, and save the registered memory length and physical segment information as the value of the B+ tree into the B+ tree data structure.

[0057] The B+ tree management unit 12 is also used to, in the case of receiving a memory cancellation request, take the virtual address as the key of the B+ tree and delete the corresponding B+ tree node.

[0058] In the implementation manner of this application, the memory access control software module 10 stores the B+ tree data structure in the form of an array of tree structure nodes and an array of content nodes. Preferably, the memory access control software module 10 saves the B+ tree data structure in the system memory.

[0059] Specifically, the B+ tree management unit 12 takes the virtual address IOVA generated by the virtual-physical address mapping unit 11 as the key of the B+ tree, and saves the length L and the physical segment information {<PAi, Offseti>|0 <= i < N} as the value of the B+ tree into the B+ tree structure. It is set that a B+ tree element can save K (K is a positive integer) physical segments, and the physical segment memory exceeding K is split and saved into multiple B+ tree elements. The B+ tree is stored in the form of a fixed-length array, including an array of tree structure nodes and an array of content nodes. The array of tree structure nodes stores the intermediate nodes and leaf nodes of the B+ tree. Among them, the intermediate node elements contain keys and pointers to the left and right subtree nodes, while the leaf node elements contain keys and pointers to the content array elements. The array of content nodes stores the complete key-value pairs. The content node elements correspond to the leaf node elements, and each content node element saves the pointer to its right sibling node element, which is convenient for range query.

[0060] Among them, the maximum number of elements contained in each B+ tree node is determined when the network card is initialized. The maximum number of physical segments contained in each B+ tree element is determined when the network card is initialized. In a memory mapping process, if the number of physical segments after the virtual address space is mapped exceeds the maximum value, these virtual addresses and physical segment information are split into multiple B+ tree elements for storage. The creation, update and deletion of the B+ tree can only be processed by the memory access control software module 10, and the processing results are synchronized to the memory access control logic module 20, so that the B+ tree virtual-physical address conversion table in the RDMA network card is consistent with the B+ tree data structure content in the system memory.

[0061] like Figure 3a The figure shows an embodiment of storing key-value pairs in a B+ tree. In this embodiment, a B+ tree element can save K=4 physical segments. The memory space registered by the application is mapped to N=3 physical segments. All physical segments can be saved in a B+ tree element. The key of the element is: IOVA, and the value is: {L, PA0, Offset0, PA1, Offset1, PA2, Offset2}.

[0062] like Figure 3b The figure shows another embodiment of storing key-value pairs in a B+ tree. In this embodiment, a B+ tree element can save K=4 physical segments. The memory space registered by the application is mapped to N=6 physical segments. These physical segments are split into two B+ tree elements for storage. The key of the first element is IOVA0, and the value is {L0, PA0, Offset0, PA1, Offset1, PA2, Offset2, PA3, Offset3}. The key of the second element is IOVA1, and the value is {L1, PA4, Offset4, PA5, Offset5}.

[0063] like Figure 3c The figure shows a schematic diagram of a B+ tree data structure provided by an exemplary example of the present application. In this example, it is assumed that a B+ tree node contains at most R=3 elements, and each element stores at most K=4 physical segments. Figure 3c A three-layer B+ tree structure is shown. It is worth noting that, since the tree structure nodes and content nodes are stored in different arrays, the leaf nodes of the B+ tree structure store the key of the element and the index pointing to the content node element corresponding to the key, and the other layers of the tree structure nodes store the key of the element and the pointer to the left and right child nodes corresponding to the key.

[0064] It should be noted that the number of nodes that the tree structure node array and the content node array can accommodate is determined when the RDMA network card driver is initialized. In some scenarios, if there are too many B+ tree nodes and the reserved array storage space is insufficient, the B+ tree management unit 12 needs to re-apply to the system for a larger array storage space, and copy all the elements in the original array to the new storage space, and finally release the original array storage space. The number of elements that each tree structure node or content node can accommodate is determined when the RDMA network card driver is initialized and will not change midway.

[0065] The memory area / memory window management unit 13 is used to perform registration and deregistration of memory areas and memory windows, generate access keys including a local access key (L_Key) and a remote access key (R_Key), and store information such as the access key, the registration address and length of the virtual address space, the address type and access rights in the memory registration list.

[0066] The operations of adding, deleting and updating B+ tree nodes by the memory access control software module 10 are all synchronized to the RDMA network card. The operations of adding, deleting and updating the memory registration list by the memory access control software module 10 are all synchronized to the RDMA network card.

[0067] In the implementation of the present application, the memory access control software module 10 stores the memory registration list in the form of an array. Preferably, the memory access control software module 10 stores the memory registration list in the system memory.

[0068] The elements of the array of the memory registration list contain: effective identification, local access key, remote access key, registration address and length of the virtual address space, address type and access rights.

[0069] In an implementation manner of the present application, the valid flag indicates whether the array element is valid. When the memory area / memory window is successfully registered, the valid flag is 1; when the memory area / memory window is deregistered, the valid flag is 0.

[0070] The local access key and remote access key are valid when the memory area / memory window has local access permission and remote access permission respectively. Both are 32-bit integers, of which the upper 24 bits are the query index of the memory registration list and the lower 8 bits are the globally unique key.

[0071] The registered address and length of the virtual address space are the starting address and total registered length of the virtual address space specified when registering a memory region or a memory window.

[0072] The address types include ordinary addresses and zero-based addresses, which are represented by the numbers 0 and 1 respectively. The access rights are divided into local write, remote read, remote write and remote atomic operation, which are represented by a 4-bit mask. It is worth noting that the address types and access rights types may be expanded with the development of the RDMA network protocol, and the address types and access rights types included in the examples of the present invention should also be expanded accordingly with the upgrade of the protocol version.

[0073] like Figure 4 The schematic diagram of a memory registration list array instance of the present application is shown. The element with array index 0 describes a valid memory area / memory window registration information, whose registered starting virtual address is 0x141200, the registration length is 10000 bytes, the address type is a normal IO address, the access rights are locally writable, remotely writable, and remotely readable, and L_Key and R_Key are 0x0A3. The element with array index 1 describes an invalid memory area / memory window registration information. Although the element contains complete registration information, the valid bit is 0, indicating that it is a set of invalid information.

[0074] In the embodiment of the present application, the memory access control logic module 20 includes a memory access permission matching logic unit 21, a virtual-physical address translation logic unit 22, a memory registration list storage unit 23 and a B+ tree storage unit 24. Among them, the memory registration list storage unit 23 is used to store a copy of the memory registration list. The B+ tree storage unit 24 is used to store a copy of the B+ tree data structure.

[0075] The memory access permission matching logic unit 21 takes the virtual address, length and key in the RDMA request as input, takes the high 24 bits of the key as the index of the memory registration list, obtains the memory registration information, and then matches the RDMA request with the information in the memory registration table. When the memory area in the RDMA request is within the memory registration range, the request key is completely equal to the registered key, and the requested read and write permissions are within the registered permission range, the memory access permission matching is successful, otherwise the permission matching fails.

[0076] As a preferred implementation, the specific implementation steps of the memory access permission matching logic unit 21 are as follows:

[0077] S211: taking the virtual address (IOVA_in), length (L_in), key (L_Key_in or R_Key_in), and access permission request (Access_in) in the RDMA request as input, taking the high 24 bits of the key as the index of the memory registration list, accessing the memory registration list storage unit 23, and obtaining a piece of memory registration information, including the aforementioned valid identifier (V), access key (L_Key, R_Key), registration address (IOVA) and length (L) of the virtual address space, address type (AType), and access permission (Access);

[0078] S212: If the valid flag V is 1, proceed to step S213, otherwise the process ends and returns a request failure;

[0079] S213: Determine whether the input key (L_Key_in or R_Key_in) is completely equal to the key (L_Key or R_Key) in the memory registration information, and whether the input access permission request (Access_in) is completely included in the access permission (Access) of the memory registration information. Only when the keys are completely equal and the access permission is completely included, proceed to step S214, otherwise terminate and return the request failure;

[0080] S214: Determine whether the memory area to be accessed in the RDMA request is within the registered range, that is, only when ((IOVA_in>=IOVA)&&(IOVA_in+L_in<=IOVA+L)) is established, it means that the memory access permission is matched successfully and the request is returned as successful, otherwise the request is returned as failed.

[0081] It is understandable that the above steps S212, S213, and S214 can be executed in parallel, and the execution results of the three steps are logically "ANDed". If the result is true, the request is returned to be successful, and if the result is false, the request fails.

[0082] In a specific embodiment, only when the memory access permission matching logic unit 21 returns a request success, the processing of the virtual-physical address translation logic unit 22 will be started. If the memory access permission matching logic unit 21 returns a request failure, the memory access control logic module 20 immediately returns a failure, and the processing of the virtual-physical address translation logic unit 22 will not be started.

[0083] Optionally, the memory registration list storage unit 23 and the B+ tree storage unit 24 can be an on-chip storage resource SRAM or an off-chip storage resource DDR. In order to reduce data access delay, on-chip storage resources are preferred.

[0084] The virtual-physical address conversion logic unit 22 uses the virtual address in the RDMA request as a key to query the B+ tree virtual-physical address conversion table. The B+ tree query is a range query. If the virtual address is found to be within the memory range represented by the length information in the key and value of a B+ tree element, the query is successful.

[0085] The specific implementation steps of the virtual-physical address translation logic unit 22 are as follows:

[0086] S221: Take the virtual address (IOVA_in) and length (L_in) in the RDMA request as input, first get the root node from the B+ tree structure node array, compare IOVA_in with the key of the root node element, and get the index of the next layer of child nodes;

[0087] S222: query each layer of the B+ tree in turn until the leaf node according to the query method of the B+ tree, query the last element that is less than or equal to IOVA_in, and obtain the content node element index corresponding to the element;

[0088] S223: Using the content node element index as the address, querying the content node array element, and obtaining a complete key-value pair, i.e., key = IOVA, value = {L, PA0, Offset0, PA1, Offset1, PA2, Offset2, PA3, Offset3}, and the right sibling pointer RightP;

[0089] S224: According to the offset of each physical segment, calculate the starting address and length of each physical segment within the memory range starting from the virtual address IOVA_in and with a length of L_in; if the expression (IOVA_in+L_in<=IOVA+L) holds, the query ends and the starting address and length of each physical segment mentioned above are output; if the expression (IOVA_in+L_in>IOVA+L) holds, it is necessary to continue to query its right sibling element, use the right sibling pointer RightP as the content node element index address, and repeat step S223.

[0090] It can be understood that, in the implementation manner of the present application, the aforementioned memory access control system for an RDMA network card is applicable to an RDMA network card device based on an Infiniband network or a RoCE network.

[0091] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A memory access control system for an RDMA network card, characterized in that: Contains a memory access control software module and a memory access control logic module; The memory access control software module is set in the host RDMA network card driver, and is used to register and deregister the memory area and the memory window, and to map the virtual address space to the physical address space; The memory access control logic module is set in the RDMA network card, and the memory access control logic module is used to determine whether the access rights requested by the RDMA are legal, and if the access rights are legal, convert the virtual address to the physical address to provide information for the DMA operation of the network card; The memory access control software module manages a memory registration list for storing registered memory areas and memory window related information; The memory access control software module also manages a B+ tree data structure for storing a mapping of virtual addresses to physical addresses; The memory access control logic module uses the storage resources in the RDMA network card hardware to save the copy of the memory registration list and the B+ tree data structure. The memory access control software module actively initiates the update of the memory registration list and the B+ tree data structure in the RDMA network card hardware to ensure the consistency of software and hardware storage; The memory access control logic module determines whether the access rights of the RDMA request are legal by accessing the memory registration list, and obtains the mapping from the virtual address to the physical address by querying the B+ tree data structure; The memory access control software module includes: A virtual-physical address mapping unit is used to map the virtual address requested by the application program to a unified IO virtual address space, and then perform virtual-physical address conversion to map a continuous virtual address space into one or more continuous physical address segments; A B+ tree management unit, used to use the registered virtual address as the key of the B+ tree, and the registered memory length and physical segmentation information as the value of the B+ tree, and save them into the B+ tree data structure; A memory area / memory window management unit, used to perform registration and deregistration of memory areas and memory windows, generate access keys including local access keys and remote access keys, and store information such as valid identifiers, access keys, registration addresses and lengths of virtual address spaces, address types, and access rights in a memory registration list; The memory access control software module stores the memory registration list in the system memory in the form of an array; The elements of the array of the memory registration list include: effective identification, local access key, remote access key, registration address and length of the virtual address space, address type and access rights; The valid flag indicates whether the array element is valid. When the memory area / memory window is successfully registered, the valid flag is 1. When the memory area / memory window is unregistered, the valid flag is 0. The local access key and remote access key are valid when the memory area / memory window has local access permission and remote access permission, respectively. Both are 32-bit integers, of which the upper 24 bits are the query index of the memory registration list and the lower 8 bits are the globally unique key. The registered address and length of the virtual address space are the starting address and total registered length of the virtual address space specified when registering the memory region or memory window; The address types include ordinary addresses and zero-based addresses, which are represented by the numbers 0 and 1 respectively; Access permissions are divided into local write, remote read, remote write and remote atomic operation, and are represented by a 4-bit mask.

2. The memory access control system for an RDMA network card according to claim 1, characterized in that: The memory access control software module stores a B+ tree data structure in the system memory in the form of a tree structure node array and a content node array; The tree structure node array stores the middle nodes and leaf nodes of the B+ tree. The middle node elements contain the key and the pointers to the left and right subtree nodes. The leaf node elements contain the key and the pointer to the content array elements. The content node array stores the complete key-value pairs. The content node elements correspond to the leaf node elements. Each content node element stores the pointer to its right sibling node element, which is convenient for range query. The maximum number of elements contained in each B+ tree node is determined when the network card is initialized; The maximum number of physical segments contained in each B+ tree element is determined when the network card is initialized; In a memory mapping process, if the number of physical segments after the virtual address space is mapped exceeds the maximum value, the virtual address and physical segment information are split into multiple B+ tree elements for storage; The creation, update and deletion of the B+ tree can only be processed by the memory access control software module, and the processing results are synchronized to the memory access control logic module, so that the B+ tree virtual-physical address conversion table in the RDMA network card is consistent with the B+ tree data structure content in the system memory.

3. The memory access control system for an RDMA network card according to claim 2, characterized in that: The memory access control logic module includes a memory access permission matching logic unit, a virtual-physical address translation logic unit, a memory registration list storage unit and a B+ tree storage unit; The memory access permission matching logic unit takes the virtual address, length and key in the RDMA request as input, takes the high 24 bits of the key as the index of the memory registration list, obtains the memory registration information, and then matches the RDMA request with the information in the memory registration table. When the memory area in the RDMA request is within the memory registration range, the request key is completely equal to the registered key, and the requested read and write permissions are within the registered permission range, the memory access permission matching is successful, otherwise the permission matching fails; The virtual-physical address conversion logic unit uses the virtual address in the RDMA request as a key to query the B+ tree virtual-physical address conversion table. The B+ tree query is a range query. If the virtual address is found to be within the memory range represented by the length information in the key and value of a B+ tree element, the query is successful. The memory registration list storage unit is used to store a copy of the memory registration list; The B+ tree storage unit is used to store a copy of the B+ tree data structure.

4. The memory access control system for an RDMA network card according to claim 1, characterized in that: The memory access control system for the RDMA network card is applicable to an RDMA network card device based on an Infiniband network or a RoCE network.

Images