Multi-node high-performance data sharing indexing method based on CXL memory architecture

Abstract

The invention discloses a multi-node high-performance data sharing index method based on a CXL memory architecture. A main server node is used for constructing a CXL data index area and a data storage area. After each server node is connected with the CXL memory device, the CXL memory device is mounted to the server node in an NUMA binding mode, and a sharing area is appointed, so that sharing access of multiple nodes to CXL physical memory data is realized. After the main server node is connected to the CXL memory, the main server node constructs a CXL data index area and a shared storage area, and supports multi-node data storage and query through a Hash index table. When a request is queried, the position of data in the hash table is positioned through the hash function, query and matching are carried out in the data hash index table, and finally the memory area where the data are located is positioned. And after the data is found, the server node loads the data from the CXL memory to the local memory for use, so that cross-server node data sharing is realized.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a multi-node high-performance data sharing indexing method based on a CXL memory architecture. Background Art

[0002] With the rapid development of big data and artificial intelligence applications, data centers are facing increasing data processing needs and memory capacity requirements. Traditional memory architectures have been unable to meet these high bandwidth and low latency requirements. Therefore, new memory technologies and architectures are constantly emerging to improve the performance and scalability of memory systems.

[0003] CXL is an open standard interconnect technology designed to achieve high-performance interconnection between computing devices through efficient cache and memory access protocols. CXL uses the PCIe physical layer to provide low-latency, high-bandwidth connections, support memory sharing and cache consistency, and enable multiple processors or accelerators to efficiently access shared memory. The introduction of the CXL memory architecture provides a new solution to the memory bottleneck problem in large-scale data centers and high-performance computing.

[0004] In traditional memory systems, data is usually distributed in the local memory of each server node, which leads to limitations in data access and complex memory management. Especially when multiple nodes need to share data, traditional memory architectures are difficult to effectively support high-speed data access across nodes and are prone to data consistency issues. Therefore, how to efficiently use CXL memory to achieve cross-node data sharing and high-performance access has become a difficult problem. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a multi-node high-performance data sharing indexing method based on the CXL memory architecture.

[0006] The present invention comprises the following steps: Step 1: Mount CXL memory to each server node, specify the shared area and establish a one-to-one mapping between CXL memory address and node virtual memory address to achieve shared access to CXL physical memory data among multiple nodes; Step 2: The master server node builds the CXL data index area and the CXL data shared storage area, and then builds the CXL data hash index table to provide support for multi-node data storage and query; Step 3: When a query request arrives, use the first hash function to calculate the hash value of the element and get the first position by modulo. If the element is not found at this position, use the second hash function to calculate the second position and check whether the position contains the element's data information; Step 4: After locating the slot where the data is located, index the memory area where the data is located; Step 5: The server node loads the memory area where this data is located from the CXL memory to the local memory for use, thereby achieving data sharing among server nodes.

[0007] Compared with the prior art, the present invention has the following beneficial effects: To address the problem of cross-node memory sharing, the present invention mounts CXL memory to each server node through NUMA binding, establishes a shared area and address mapping, and realizes efficient memory sharing across nodes, thereby simplifying the system architecture, reducing latency, and improving overall performance.

[0008] To address the problem of low query efficiency, the present invention adopts a CXL data hash index table, optimizes hash functions and index structures, reduces hash conflicts and avoids the use of linked list structures, reduces the frequency of access to the CXL memory, improves query efficiency, and reduces latency.

[0009] In order to solve the problem of low utilization rate of CXL memory when loading, the present invention reduces the memory space occupied by address pointers by using metadata fields and address offsets, improves memory utilization rate, and realizes efficient management and accurate positioning of large-scale data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical scheme and advantages of the present invention clearer, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. Examples of these preferred embodiments are illustrated in the accompanying drawings. The embodiments of the present invention shown in the accompanying drawings and described according to the accompanying drawings are merely exemplary, and the present invention is not limited to these embodiments.

[0012] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0013] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0014] In traditional memory architectures, achieving memory sharing between different server nodes is a complex and inefficient task. In a multi-node system, data transmission and access between nodes usually rely on network communication, which not only increases latency but also reduces the overall performance of the system. How to achieve efficient memory sharing across nodes has become an important technical problem.

[0015] In large-scale data storage, how to efficiently manage and quickly query data is a major challenge. Traditional data management methods are prone to hash conflicts and uneven data distribution when processing large-scale data, resulting in low query efficiency. Especially in the hash index storage process, hash conflicts are inevitable, and using linked lists to alleviate conflicts will lead to frequent access to CXL memory. Compared with the server's local memory, CXL memory has a higher access latency, so it is necessary to avoid using linked list structures.

[0016] In traditional memory management methods, data address pointers occupy a large amount of memory, resulting in low utilization when CXL memory is loaded. In large-scale data storage, efficient use of memory is crucial. It is necessary to design a memory management method that can accurately locate data, reduce the memory space occupied by address pointers, and improve memory utilization.

[0017] Based on the above-mentioned deficiencies in the prior art, the present invention provides a multi-node high-performance data sharing indexing method based on the CXL memory architecture. The present invention is based on the multi-server nodes shared by the CXL memory, which are divided into master server nodes and slave server nodes. The master server node is used to build the CXL data index area and data storage area. After each server node is connected to the CXL memory device, the CXL memory device is mounted to the server node through the NUMA binding method, and the shared area is specified to realize multi-node shared access to the CXL physical memory data. After connecting to the CXL memory, the master server node builds the CXL data index area and the shared storage area, and supports multi-node data storage and query through the hash index table. When the query request is made, the position of the data in the hash table is located by the hash function, and the query and match are performed in the data hash index table, and finally the memory area where the data is located is located. After finding the data, the server node loads the data from the CXL memory into the local memory for use, thereby realizing data sharing across server nodes.

[0018] The specific process of the present invention is as follows Figure 1 As shown, the following steps are included: Step 1: CXL memory is mounted to each server node through NUMA binding, and then a shared area is specified and a one-to-one mapping between CXL memory addresses and node virtual memory addresses is established to achieve shared access to CXL physical memory data among multiple nodes.

[0019] When a CXL memory is connected to the server motherboard through a PCIe slot, the CXL memory will be managed. At this time, the CXL memory device will be mounted on the server node through NUMA binding. After binding, the CXL memory device will be regarded as a NUMA node of the server node, and the server node can directly access the CXL memory device through the memory controller in its NUMA architecture.

[0020] Furthermore, in order to share data on the CXL physical memory between multiple server nodes, the CXL device memory is divided and a certain memory area is designated as a shared area. After the CXL memory device is connected to the server node, a one-to-one mapping relationship between the CXL physical memory address and the server node virtual memory address needs to be established for the memory in this area. First, the CXL memory is mapped to the physical address space of the server node. Next, these physical addresses are directly mapped to a fixed virtual address space of the server node. At the same time, for this memory space, a set of CXL memory allocators is re-established to make the CXL memory and the local memory allocation method of the server node different, so as to facilitate direct data access to the CXL memory. When this shared memory is actually used, the server node accesses data through its local virtual memory address, and the request will be converted by the operating system into an access operation to the CXL physical memory. This allows each server node to access the CXL data index area and CXL data shared storage area established on the CXL memory in step 2.

[0021] Step 2: After connecting to the CXL memory, the master server node builds the CXL data index area and the CXL data shared storage area, and then builds the CXL data hash index table to provide support for multi-node data storage and query.

[0022] CXL memory needs to be physically connected to the server node manually, so for the server that is first connected to the CXL memory device, this application will establish it as the main server node. After connecting to the CXL memory, the main server node will immediately build the CXL data index area and CXL data shared storage area on the CXL memory device. The CXL data index area is allocated from the low address space of the shared area; the CXL data shared storage area is opened up from the high address space of the shared area, mainly used to store actual data content. The length of the CXL data index area is not fixed, and it will be expanded in the future as the amount of data increases.

[0023] Then, the CXL data hash index table is constructed, which consists of a fixed-size array. Each array element occupies 64 bytes of memory space and exists as a bucket. Each bucket contains four slots, and one slot occupies 8 bytes of memory space and exists as a data item. When a hash conflict occurs, the role of the slot is to alleviate this operation. The memory space in the slot does not store the actual content of the data, but stores the metadata information of the data. At this time, all buckets and slots in the hash table will be allocated memory and initialized to empty or NULL. The size of the hash table is usually pre-set and will be decided by the user. After the CXL data hash index table is constructed, all server nodes insert and store data into it. When data is stored, the data will first be stored in the CXL data storage area through the CXL memory allocator, and then the data metadata will be inserted into the CXL data hash index table. This application adopts a data insertion method similar to the cuckoo hash table. This process can realize data storage operations, which is convenient for subsequent nodes to perform query requests.

[0024] Step 3: When a query request arrives, the first hash function is used to calculate the hash value of the element, and the first position is obtained by modulo. If the element is not found at this position, the second hash function is used to calculate the second position and check whether the position contains the element's data information. Specifically: When a query request arrives, for an element data x that needs to be searched, first use the first hash function h1 to calculate the hash value h1(x) of the element x relative to the hash function h1. If the number of bucket items in the CXL data hash index table is n, the first position in the hash table is obtained by modulo p1(x) = h1(x) mod n.

[0025] After calculating the position p1(x), read the 64-byte bucket memory at this index position to the local DRAM memory of the server node, and then check the 4 slot items stored therein in turn. The first metadata on each slot item is the identification field, which is the hash value f(x) of element x according to the fingerprint function f. It will be compressed into one byte and used to identify the slot corresponding to element x. This application will calculate the identification field of element x in advance and match it with the four slots on the bucket. If the match is successful, it means that element x can be located. If element x is not found in this bucket, it means that element x either does not exist in the CXL data hash index table or is stored in another location.

[0026] In order to further confirm whether element x exists, the first position information and fingerprint function are used to perform operations to locate the second position p2(x). The specific process is p2(x) = (p1(x) xor h2(f(x))) mod n, where h2 is the second hash function. Then the second position on the table is accessed, the bucket memory is obtained, and the slot on the bucket is checked to see if it contains the data information of element x. This involves another memory read operation. If element x is found at this position, step 4 will be performed. If element x is still not found in the second bucket, it can be determined that element x does not exist in the table. At this point, the query process ends and the result not found is returned.

[0027] Step 4: After locating the slot where the data is located, the memory area where the data is located will be indexed.

[0028] When the slot where the requested data element is located is found, a slot consists of two parts, namely the identification field and the metadata field. The identification field is a 1-byte hash value corresponding to the fingerprint function; the metadata field contains a 4-byte data address offset and a 3-byte number of memory pages occupied by the data. The 4-byte data address offset will be combined with the starting address of the CXL data shared storage area to form a new address, thereby truly locating the memory address of the data element in the memory.

[0029] Furthermore, in order to avoid the data address pointer occupying too many bytes, resulting in excessive bucket memory usage, the CXL data shared storage area will open up a continuous memory space to store data elements, and the CXL address of the data element will also consist of the data address offset and the starting address of the CXL data shared storage area. Each data element is stored in pages, and the number of 3-byte data memory pages will indicate the memory space that needs to be loaded. Then, according to the number of memory pages in the slot, the memory area where the data is stored will be determined.

[0030] Step 5: The server node loads the memory area where this data is located from the CXL memory to the local memory for use, thus achieving data sharing among server nodes.

[0031] It should be noted that, in this specification, "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0032] It should be understood that although this specification is described according to implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A multi-node high-performance data sharing indexing method based on CXL memory architecture, characterized in that The method comprises the following steps: Step 1: Mount CXL memory to each server node, specify the shared area and establish a one-to-one mapping between CXL memory address and node virtual memory address to achieve shared access to CXL physical memory data among multiple nodes; Step 2: The master server node builds the CXL data index area and the CXL data shared storage area, and then builds the CXL data hash index table to provide support for multi-node data storage and query; Step 3: When a query request arrives, use the first hash function to calculate the hash value of the element and get the first position by modulo. If the element is not found at this position, use the second hash function to calculate the second position and check whether the position contains the element's data information. Step 4: After locating the slot where the data is located, index the memory area where the data is located; Step 5: The server node loads the memory area where this data is located from the CXL memory to the local memory for use, thereby achieving data sharing among server nodes.

2. The multi-node high-performance data sharing indexing method according to claim 1 is characterized in that: The CXL memory described in step 1 is mounted to each server node through NUMA binding.

3. The multi-node high-performance data sharing indexing method according to claim 1 is characterized in that: The specific steps of establishing a one-to-one mapping between CXL memory addresses and node virtual memory addresses in step 1 are: Map CXL memory to the physical address space of the server node; The physical address is directly mapped to a fixed virtual address space of the server node.

4. The multi-node high-performance data sharing indexing method according to claim 1 is characterized in that: In step 2, the CXL data index area is allocated from the low address space of the shared area; and the CXL data shared storage area is opened from the high address space of the shared area.

5. The multi-node high-performance data sharing indexing method according to claim 1 is characterized in that: The hash index table described in step 2 is composed of an array of a fixed size, each array element occupies 64 bytes of memory space and exists as a storage bucket; each storage bucket contains four slots, each slot occupies 8 bytes of memory space and exists as a data item.

6. The multi-node high-performance data sharing indexing method according to claim 1 or 5, characterized in that: The second position described in step 3 is also related to the first position.

7. The multi-node high-performance data sharing indexing method according to claim 1 is characterized in that: In step 4, the slot where the data is located includes two parts, namely, the identification field and the metadata field; The identification field is a hash value corresponding to a 1-byte fingerprint function; The metadata field contains a 4-byte data address offset and a 3-byte number of memory pages occupied by the data; the 4-byte data address offset and the starting address of the CXL data shared storage area are combined and spliced ​​into a new address, thereby truly locating the memory address of the data element in the memory.

8. The multi-node high-performance data sharing indexing method according to claim 7, characterized in that: The data elements are stored in pages, and the number of 3-byte data memory pages will indicate the memory space that needs to be loaded; based on the number of memory pages in the slot, the memory area where the data is stored is determined.

Images