Distributed database system based on RDMA shared memory pool

Abstract

The invention discloses a distributed database system based on an RDMA shared memory pool, and relates to the technical field of distributed data management. According to the method, the two database cluster nodes are set as the shared memory nodes, the shared memory nodes are the main and standby nodes, the shared memory pool is created through the physical memory of the shared memory nodes, the metadata are stored in a classified manner, the copy frequency and kernel intervention in the data transmission process are reduced by utilizing the low-delay characteristic of the RDMA, the utilization rate of a CPU and the memory is reduced, and the data transmission efficiency is improved. And the read-write time and the query response time of the data are greatly shortened. According to the scheme, the performance of the distributed database in a high-concurrency access scene is remarkably improved.

Description

Technical Field

[0001] The present invention discloses a distributed database system based on RDMA shared memory pool, and relates to the technical field of distributed data management. Background Art

[0002] Shared nothing architecture is a common distributed database architecture. Each node has its own private CPU, memory, hard disk, etc. There are no shared resources. The processing units communicate through protocols, and the parallel processing and expansion capabilities are better. Each node is independent of each other and processes its own data. The processed results may be summarized to the upper layer or circulated between nodes. The shared nothing distributed database system continuously broadcasts the latest metadata to other nodes in the cluster through the data synchronization protocol. In the process of node metadata synchronization, the following problems may exist. As the cluster scale expands, the two farthest nodes in the cluster need to go through multiple intermediate nodes when initiating data synchronization, resulting in data synchronization delays. When the intermediate node network is poor, data loss may even occur, affecting the high availability and high concurrency characteristics of the distributed database system. Summary of the invention

[0003] In view of the problems of the prior art, the present invention provides a distributed database system based on RDMA shared memory pool, which greatly improves the operation efficiency and response speed of the database, reduces system resource consumption, and improves the performance of large-scale data processing and high concurrent access.

[0004] The specific scheme proposed by the present invention is:

[0005] The present invention provides a distributed database system based on RDMA shared memory pool, including a node management module, an RDMA memory management module, a data transmission and synchronization module and a query processing module.

[0006] The node management module selects two nodes in the distributed database cluster as shared memory nodes, designates one shared memory node as the primary node and the other shared memory node as the backup node.

[0007] The RDMA memory management module applies for and locks the physical memory of the shared memory node to create a shared memory pool based on the estimated database cluster node scale and user data capacity, and divides the node shared memory and data table shared memory in the shared memory pool to store different types of metadata.

[0008] Enable the master node and backup node to provide external services through the listening address of the shared memory pool;

[0009] Initialize the shared memory pool: Before the distributed database cluster is started, the data transmission and synchronization module of the master node starts listening, the RDMA memory management module of the master node generates the corresponding shared memory pool memory and permission information, the data transmission and synchronization module of the backup node initiates an RDMA connection to the master node, obtains the shared memory pool memory and permission information of the master node, and at the same time, the RDMA memory management module of the backup node generates the shared memory pool memory and permission information of the same specifications;

[0010] The data transmission and synchronization modules of other database cluster nodes actively initiate RDMA connections to the shared memory pool listening address, and initiate information synchronization operations to obtain the memory and permission information of the shared memory pool. During the operation of the database cluster nodes, the RDMA memory management module of the database cluster nodes writes the newly generated metadata or the changed metadata into the shared memory pool through RDMA write operations;

[0011] The random access nodes in the distributed database cluster receive query requests through the query processing module. The query processing module builds a distributed logical plan and physical plan. According to the logical plan and physical plan, the random access nodes use the data transmission and synchronization module to connect to the master node through RDMA and read the corresponding information from the shared memory pool.

[0012] Furthermore, in the distributed database system based on RDMA shared memory pool, permission information is required when the shared memory pool performs RDMA read and write operations. The data transmission and synchronization module of the master node synchronizes the permission information to other distributed database cluster nodes for reading and writing memory. At the same time, the master node and the backup node synchronize the permission information with each other through the data transmission and synchronization module. After the backup node switches to the new master node, the permission information is synchronized to other distributed database cluster nodes.

[0013] Furthermore, the RDMA memory management module of the master node in the distributed database system based on RDMA shared memory pool divides the shared memory pool into multiple memory blocks of fixed size, and records the status and location information of each memory block.

[0014] When the database cluster node initiates a memory write operation, the RDMA memory management module obtains the write address according to the address calculation rules and writes the metadata to the shared memory pool. When the database cluster node initiates a memory read operation, the RDMA memory management module reads the data response from the corresponding memory block according to the memory virtual address and offset.

[0015] Furthermore, when the RDMA memory management module in the distributed database system based on the RDMA shared memory pool writes metadata into the shared memory pool, the types of metadata include node configuration information, node health status, node storage usage and table information, and are assigned numbers 0-4 in sequence, or new numbers are assigned to metadata of new types as needed.

[0016] Furthermore, in the distributed database system based on RDMA shared memory pool, the RDMA memory management module sequentially generates each node ID of the database cluster starting from sequence 1, and the RDMA memory management module obtains the address block of the metadata of the database cluster node written into the shared memory pool by calculating the node shared memory first address + (node ​​ID-1) * metadata quantity + metadata number, and obtains the write address of the data table by calculating the data table shared memory first address + table ID.

[0017] The present invention also provides a distributed database management method based on RDMA shared memory pool, which performs distributed database management through a distributed database system based on RDMA shared memory pool. The distributed database system includes a node management module, an RDMA memory management module, a data transmission and synchronization module, and a query processing module.

[0018] Use the node management module to select two nodes in the distributed database cluster as shared memory nodes, designate one shared memory node as the primary node and the other shared memory node as the backup node.

[0019] The RDMA memory management module applies for and locks the physical memory of the shared memory node according to the estimated database cluster node scale and user data capacity to create a shared memory pool, and divides the node shared memory and data table shared memory in the shared memory pool to store different types of metadata.

[0020] Enable the master node and backup node to provide external services through the listening address of the shared memory pool;

[0021] Initialize the shared memory pool: Before the distributed database cluster is started, the data transmission and synchronization module of the master node is used to start listening, and the RDMA memory management module of the master node is used to generate the corresponding shared memory pool memory and permission information. The data transmission and synchronization module of the backup node initiates an RDMA connection to the master node to obtain the shared memory pool memory and permission information of the master node. At the same time, the RDMA memory management module of the backup node is used to generate the shared memory pool memory and permission information of the same specifications.

[0022] The data transmission and synchronization modules of other database cluster nodes actively initiate an RDMA connection to the shared memory pool listening address, and initiate an information synchronization operation to obtain the memory and permission information of the shared memory pool. During the operation of the database cluster node, the RDMA memory management module of the database cluster node writes the newly generated metadata or the changed metadata into the shared memory pool through the RDMA write operation;

[0023] The random access nodes in the distributed database cluster receive query requests through the query processing module. The query processing module builds a distributed logical plan and physical plan. According to the logical plan and physical plan, the random access nodes use the data transmission and synchronization module to connect to the master node through RDMA and read the corresponding information from the shared memory pool.

[0024] Furthermore, the distributed database management method based on RDMA shared memory pool is as follows: when performing RDMA read and write operations in the shared memory pool, permission information is required, and the permission information is synchronized to other distributed database cluster nodes for reading and writing memory through the data transmission and synchronization module of the master node. At the same time, the master node and the backup node synchronize the permission information with each other through the data transmission and synchronization module, and the backup node synchronizes the permission information to other distributed database cluster nodes after switching to the new master node.

[0025] Furthermore, the distributed database management method based on RDMA shared memory pool is as follows: the shared memory pool is divided into a plurality of fixed-size memory blocks by the RDMA memory management module of the master node, and the status and location information of each memory block is recorded,

[0026] When the database cluster node initiates a memory write operation, the RDMA memory management module obtains the write address according to the address calculation rules and writes the metadata to the shared memory pool. When the database cluster node initiates a memory read operation, the RDMA memory management module reads the data response from the corresponding memory block according to the memory virtual address and offset.

[0027] Furthermore, the distributed database management method based on RDMA shared memory pool: when metadata is written into the shared memory pool through the RDMA memory management module, the types of metadata include node configuration information, node health status, node storage usage and table information, and are assigned numbers 0-4 in sequence, or new numbers are assigned to metadata of new types as needed.

[0028] Furthermore, the distributed database management method based on RDMA shared memory pool is as follows: the RDMA memory management module sequentially generates each node ID of the database cluster starting from sequence 1, the RDMA memory management module obtains the address block of the metadata of the database cluster node written into the shared memory pool by calculating the node shared memory first address + (node ​​ID-1) * metadata quantity + metadata number, and obtains the write address of the data table by calculating the data table shared memory first address + table ID.

[0029] The benefits of the present invention are:

[0030] Set two database cluster nodes as shared memory nodes, with the shared memory nodes being the primary and standby nodes. Create a shared memory pool through the physical memory of the shared memory nodes, and store metadata in a classified manner. Utilize the low latency feature of RDMA to reduce the number of copies and kernel intervention during data transmission, reduce CPU and memory usage, and greatly shorten data read and write time and query response time. This solution significantly improves the performance of distributed databases in high-concurrency access scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the distributed database cluster architecture based on RDMA shared memory pool.

[0032] Figure 2 It is a flow chart of the method of the present invention using a shared memory pool.

[0033] Figure 3 It is a schematic diagram of the query request process executed by the method of the present invention. DETAILED DESCRIPTION

[0034] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0035] Example 1

[0036] The present invention provides a distributed database system based on RDMA shared memory pool, including a node management module, an RDMA memory management module, a data transmission and synchronization module and a query processing module.

[0037] The node management module selects two nodes in the distributed database cluster as shared memory nodes, designates one shared memory node as the primary node and the other shared memory node as the backup node, and automatically switches to the backup node as the new primary node when the current primary node fails.

[0038] The RDMA memory management module applies for and locks the physical memory of the shared memory node to create a shared memory pool based on the estimated database cluster node scale and user data capacity, and divides the node shared memory and data table shared memory in the shared memory pool to store different types of metadata. At the same time, when applying for physical memory, it generates a mapping relationship between virtual addresses and physical addresses, and provides the virtual addresses to applications.

[0039] The master node and backup node provide external services through the listening address of the shared memory pool.

[0040] The RDMA memory management module of the master node divides the shared memory pool into multiple fixed-size memory blocks and records the status and location information of each memory block.

[0041] When the database cluster node initiates a memory write operation, the RDMA memory management module obtains the write address according to the address calculation rules and writes the metadata to the shared memory pool. When the database cluster node initiates a memory read operation, the RDMA memory management module reads the data response from the corresponding memory block according to the memory virtual address and offset. In the shared memory pool, the master and backup nodes apply for memory blocks of the same size and divide them consistently. The mapping relationship between the memory block addresses is stored between the master and backup nodes. When the data of a memory block of the master node changes, it will be synchronized to the backup node.

[0042] Initialize the shared memory pool: Before the distributed database cluster is started, the data transmission and synchronization module of the master node starts listening, the RDMA memory management module of the master node generates the corresponding shared memory pool memory and permission information, the data transmission and synchronization module of the backup node initiates an RDMA connection to the master node, obtains the shared memory pool memory and permission information of the master node, and at the same time, the RDMA memory management module of the backup node generates the shared memory pool memory and permission information of the same specifications.

[0043] The data transmission and synchronization modules of other database cluster nodes actively initiate RDMA connections to the shared memory pool listening address, and initiate information synchronization operations to obtain the memory and permission information of the shared memory pool, such as memory address information, memory size and block information, permission information, etc. During the operation of the database cluster nodes, the RDMA memory management module of the database cluster nodes writes the newly generated metadata or changed metadata into the shared memory pool through RDMA write operations. In the shared memory pool, permission information is required when performing RDMA read and write operations. The data transmission and synchronization module of the master node synchronizes the permission information to other distributed database cluster nodes for reading and writing memory. At the same time, the master node and the backup node synchronize the permission information with each other through the data transmission and synchronization module. After the backup node switches to the new master node, the permission information is synchronized to other distributed database cluster nodes.

[0044] When the RDMA memory management module writes metadata into the shared memory pool, the types of metadata include node configuration information, node health status, node storage usage, and table information, and are assigned numbers 0-4 in sequence, or new numbers are assigned as needed to expand new types of metadata. The RDMA memory management module generates each node ID of the database cluster sequentially starting from sequence 1. The RDMA memory management module obtains the address block where the metadata of the database cluster node is written into the shared memory pool through the node shared memory first address + (node ​​ID-1) * metadata quantity + metadata number calculation, and obtains the write address of the data table through the data table shared memory first address + table ID calculation.

[0045] The random access nodes in the distributed database cluster receive query requests through the query processing module. The query processing module builds a distributed logical plan and physical plan. According to the logical plan and physical plan, the random access nodes use the data transmission and synchronization module to connect to the master node through RDMA and read the corresponding information from the shared memory pool.

[0046] Example 2

[0047] The present invention also provides a distributed database management method based on RDMA shared memory pool, which performs distributed database management through a distributed database system based on RDMA shared memory pool. The distributed database system includes a node management module, an RDMA memory management module, a data transmission and synchronization module, and a query processing module.

[0048] Use the node management module to select two nodes in the distributed database cluster as shared memory nodes, designate one shared memory node as the primary node and the other shared memory node as the backup node.

[0049] The RDMA memory management module applies for and locks the physical memory of the shared memory node according to the estimated database cluster node scale and user data capacity to create a shared memory pool, and divides the node shared memory and data table shared memory in the shared memory pool to store different types of metadata.

[0050] Enable the master node and backup node to provide external services through the listening address of the shared memory pool;

[0051] Initialize the shared memory pool: Before the distributed database cluster is started, the data transmission and synchronization module of the master node is used to start listening, and the RDMA memory management module of the master node is used to generate the corresponding shared memory pool memory and permission information. The data transmission and synchronization module of the backup node initiates an RDMA connection to the master node to obtain the shared memory pool memory and permission information of the master node. At the same time, the RDMA memory management module of the backup node is used to generate the shared memory pool memory and permission information of the same specifications.

[0052] The data transmission and synchronization modules of other database cluster nodes actively initiate an RDMA connection to the shared memory pool listening address, and initiate an information synchronization operation to obtain the memory and permission information of the shared memory pool. During the operation of the database cluster node, the RDMA memory management module of the database cluster node writes the newly generated metadata or the changed metadata into the shared memory pool through the RDMA write operation;

[0053] The random access nodes in the distributed database cluster receive query requests through the query processing module. The query processing module builds a distributed logical plan and physical plan. According to the logical plan and physical plan, the random access nodes use the data transmission and synchronization module to connect to the master node through RDMA and read the corresponding information from the shared memory pool.

[0054] The information interaction, execution process and other contents involved in the above method flow are based on the same concept as the system embodiment of the present invention. For specific contents, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0055] Similarly, the method of the present invention sets two database cluster nodes as shared memory nodes, the shared memory nodes are primary and standby nodes, creates a shared memory pool through the physical memory of the shared memory node, and stores metadata in a classified manner, using the low latency characteristics of RDMA, reducing the number of copies and kernel intervention during data transmission, reducing the utilization rate of CPU and memory, and greatly shortening the data reading and writing time and query response time. This solution significantly improves the performance of distributed databases in high concurrent access scenarios.

[0056] It should be noted that not all steps and modules in the above-mentioned system structures and processes are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0057] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A distributed database system based on RDMA shared memory pool, characterized by It includes node management module, RDMA memory management module, data transmission and synchronization module and query processing module. The node management module selects two nodes in the distributed database cluster as shared memory nodes, designates one shared memory node as the primary node and the other shared memory node as the backup node. The RDMA memory management module applies for and locks the physical memory of the shared memory node to create a shared memory pool based on the estimated database cluster node scale and user data capacity, and divides the node shared memory and data table shared memory in the shared memory pool to store different types of metadata. Enable the master node and backup node to provide external services through the listening address of the shared memory pool; Initialize the shared memory pool: Before the distributed database cluster is started, the data transmission and synchronization module of the master node starts listening, the RDMA memory management module of the master node generates the corresponding shared memory pool memory and permission information, the data transmission and synchronization module of the backup node initiates an RDMA connection to the master node, obtains the shared memory pool memory and permission information of the master node, and at the same time, the RDMA memory management module of the backup node generates the shared memory pool memory and permission information of the same specifications; The data transmission and synchronization modules of other database cluster nodes actively initiate RDMA connections to the shared memory pool listening address, and initiate information synchronization operations to obtain the memory and permission information of the shared memory pool. During the operation of the database cluster nodes, the RDMA memory management module of the database cluster nodes writes the newly generated metadata or the changed metadata into the shared memory pool through RDMA write operations; The random access nodes in the distributed database cluster receive query requests through the query processing module. The query processing module builds a distributed logical plan and physical plan. According to the logical plan and physical plan, the random access nodes use the data transmission and synchronization module to connect to the master node through RDMA and read the corresponding information from the shared memory pool.

2. A distributed database system based on RDMA shared memory pool according to claim 1, characterized in that The shared memory pool requires permission information when performing RDMA read and write operations. The data transmission and synchronization module of the master node synchronizes the permission information to other distributed database cluster nodes for reading and writing memory. At the same time, the master node and the backup node synchronize the permission information with each other through the data transmission and synchronization module. After the backup node switches to the new master node, the permission information is synchronized to other distributed database cluster nodes.

3. A distributed database system based on RDMA shared memory pool according to claim 1, characterized in that The RDMA memory management module of the master node divides the shared memory pool into multiple fixed-size memory blocks and records the status and location information of each memory block. When the database cluster node initiates a memory write operation, the RDMA memory management module obtains the write address according to the address calculation rules and writes the metadata to the shared memory pool. When the database cluster node initiates a memory read operation, the RDMA memory management module reads the data response from the corresponding memory block according to the memory virtual address and offset.

4. A distributed database system based on RDMA shared memory pool according to claim 1, characterized in that When the RDMA memory management module writes metadata into the shared memory pool, the types of metadata include node configuration information, node health status, node storage usage, and table information, and are assigned numbers 0-4 in sequence, or new numbers are assigned to metadata of new types as needed.

5. The distributed database system based on RDMA shared memory pool according to claim 1, characterized in that The RDMA memory management module generates each node ID of the database cluster sequentially starting from sequence 1. The RDMA memory management module obtains the address block where the metadata of the database cluster node is written into the shared memory pool by calculating the node shared memory first address + (node ​​ID-1) * metadata quantity + metadata number, and obtains the write address of the data table by calculating the data table shared memory first address + table ID.

6. A distributed database management method based on RDMA shared memory pool, characterized in that Distributed database management is performed through a distributed database system based on RDMA shared memory pool. The distributed database system includes a node management module, an RDMA memory management module, a data transmission and synchronization module, and a query processing module. Use the node management module to select two nodes in the distributed database cluster as shared memory nodes, designate one shared memory node as the primary node and the other shared memory node as the backup node. The RDMA memory management module applies for and locks the physical memory of the shared memory node according to the estimated database cluster node scale and user data capacity to create a shared memory pool, and divides the node shared memory and data table shared memory in the shared memory pool to store different types of metadata. Enable the master node and backup node to provide external services through the listening address of the shared memory pool; Initialize the shared memory pool: Before the distributed database cluster is started, the data transmission and synchronization module of the master node is used to start listening, and the RDMA memory management module of the master node is used to generate the corresponding shared memory pool memory and permission information. The data transmission and synchronization module of the backup node initiates an RDMA connection to the master node to obtain the shared memory pool memory and permission information of the master node. At the same time, the RDMA memory management module of the backup node is used to generate the shared memory pool memory and permission information of the same specifications. The data transmission and synchronization modules of other database cluster nodes actively initiate an RDMA connection to the shared memory pool listening address, and initiate an information synchronization operation to obtain the memory and permission information of the shared memory pool. During the operation of the database cluster node, the RDMA memory management module of the database cluster node writes the newly generated metadata or the changed metadata into the shared memory pool through the RDMA write operation; The random access nodes in the distributed database cluster receive query requests through the query processing module. The query processing module builds a distributed logical plan and physical plan. According to the logical plan and physical plan, the random access nodes use the data transmission and synchronization module to connect to the master node through RDMA and read the corresponding information from the shared memory pool.

7. The distributed database management method based on RDMA shared memory pool according to claim 6, characterized in that Permission information is required for RDMA read and write operations in the shared memory pool. The permission information is synchronized to other distributed database cluster nodes through the data transmission and synchronization module of the master node for reading and writing memory. At the same time, the master node and the backup node synchronize permission information with each other through the data transmission and synchronization module. After the backup node switches to the new master node, the permission information is synchronized to other distributed database cluster nodes.

8. The distributed database management method based on RDMA shared memory pool according to claim 6 is characterized by: The RDMA memory management module of the master node divides the shared memory pool into multiple fixed-size memory blocks and records the status and location information of each memory block. When the database cluster node initiates a memory write operation, the RDMA memory management module obtains the write address according to the address calculation rules and writes the metadata to the shared memory pool. When the database cluster node initiates a memory read operation, the RDMA memory management module reads the data response from the corresponding memory block according to the memory virtual address and offset.

9. The distributed database management method based on RDMA shared memory pool according to claim 6, characterized in that When metadata is written to the shared memory pool through the RDMA memory management module, the types of metadata include node configuration information, node health status, node storage usage, and table information, and are assigned numbers 0-4 in sequence, or new numbers are assigned if new types of metadata are expanded as needed.

10. The distributed database management method based on RDMA shared memory pool according to claim 6, characterized in that The RDMA memory management module generates each node ID of the database cluster sequentially starting from sequence 1. The RDMA memory management module obtains the address block where the metadata of the database cluster node is written into the shared memory pool by calculating the node shared memory first address + (node ​​ID-1) * metadata quantity + metadata number, and obtains the write address of the data table by calculating the data table shared memory first address + table ID.

Images