Database cluster system and method based on Linux lightweight virtualization technology

Abstract

The invention aims to provide a database cluster system and method based on a Linux lightweight virtualization technology. The database cluster system based on the Linux lightweight virtualization technology comprises a physical layer, a database cluster layer, a database cluster layer and a database cluster layer, wherein the physical layer is provided with standard x86 server hardware; the operating system layer is provided with a Linux operating system which supports cgroup and LXC functions; the virtualization layer is used for creating an independent resource control group for each application or a key component thereof based on a cgroup function of a Linux kernel, and accurately controlling a CPU time slice, a memory usage and recovery strategy, a block device I / O bandwidth and weight, and a network bandwidth and priority; and the application layer is a key big data application component which comprises an FE / BE node of a database service Apache Dores, a calculation engine and a Web service which are respectively operated in independent LXC containers. According to the method, efficient utilization of hardware resources, strong isolation between applications, stability guarantee, rapid elastic expansion of the resources and smooth seamless evolution from a single-machine all-in-one machine to a distributed cluster can be realized, so that a hardware input curve is smoother.

Description

TECHNICAL FIELD

[0001] The application relates to a database cluster system based on a Linux light virtualization technology, and relates to a database cluster method based on the Linux light virtualization technology, and belongs to the database field. BACKGROUND

[0002] Currently, enterprises face the following main pain points when deploying and running big data applications such as data analysis, real-time reporting, BI and the like:

[0003] 1. High hardware cost and steep investment curve: in the traditional deployment mode - single machine or static cluster, in order to meet the peak load or future growth demand, excessive hardware resources need to be pre-installed, which leads to huge initial investment and low utilization of equipment in the off-peak period, and the hardware cost is high.

[0004] 2. Poor stability and resource conflict: when deploying multiple applications on a single physical server, there is a lack of effective resource isolation mechanism, which leads to the application performance instability and even crash caused by the application contention for CPU, memory, IO, network and other resources.

[0005] 3. Low resource allocation efficiency and inflexible expansion:

[0006] Virtual machine (VM) technology: although it provides resource isolation, each VM needs to run a complete operating system kernel, which leads to large resource overhead, slow startup, relatively cumbersome expansion / shrinking operation and low resource allocation efficiency.

[0007] Hyper-converged architecture (HCI), Kubernetes (K8s) and the like: provide strong flexibility and management capability, but the underlying architecture is complex, and multiple layers of virtualization and management components are introduced: such as KVM / ESXi + distributed storage + K8s control plane, which leads to significant performance loss, especially for I / O intensive applications, and the learning and operation cost is high, which is not suitable for performance sensitive or lightweight scenarios.

[0008] 4. Lack of smooth architecture evolution capability: when the business of an enterprise grows, it often needs to expand from a single machine to a distributed cluster. The traditional solution switching process is complex, involves data migration, configuration change and application adaptation, and it is difficult to realize seamless switching. SUMMARY

[0009] The application aims to provide a database cluster system and method based on a Linux light virtualization technology to solve the above problems,

[0010] The application adopts the following technical solutions:

[0011] A database cluster system based on Linux lightweight virtualization technology, characterized in that it includes:

[0012] Physical layer: Features standard x86 server hardware;

[0013] Operating system layer: A Linux operating system that supports cgroups and LXC;

[0014] Virtualization layer: Based on the cgroups function of the Linux kernel, it creates an independent resource control group for each application or its critical components, and precisely controls CPU time slices, memory usage and reclamation policies, block device I / O bandwidth and weight, and network bandwidth and priority;

[0015] Application Layer: Key big data application components, including the FE / BE nodes of the Apache Doris database service, the computing engine, and the web services, all running in independent LXC containers.

[0016] Furthermore, the database cluster system based on Linux lightweight virtualization technology of the present invention also has the following features:

[0017] Includes a configuration management module: storage and management system deployment modes: single machine or cluster, node information, and application configuration.

[0018] Furthermore, the database cluster system based on Linux lightweight virtualization technology of the present invention also has the following features:

[0019] In standalone mode: Doris FE, Doris BE, and the default application all run in their own independent containers on the same physical node.

[0020] Furthermore, the database cluster system based on Linux lightweight virtualization technology of the present invention also has the following features:

[0021] It includes a monitoring and scheduling module to monitor the resource usage of the host machine and containers, including CPU, Mem, IO, Net, and database cluster load and status.

[0022] Furthermore, the database cluster system based on Linux lightweight virtualization technology of the present invention also has the following features:

[0023] It has a metadata management module that, based on preset policies or administrator commands, is responsible for parsing and executing operations that record the address and port information of the new BE into the cluster's metadata, thus completing the expansion operation.

[0024] Furthermore, the database cluster system based on Linux lightweight virtualization technology of the present invention also has the following features:

[0025] The metadata management module is used to quickly deploy new containers, including Doris BE, on new physical nodes via LXC when it receives an instruction to expand to cluster mode. It also calls the Doris management interface to dynamically add the new BE node to the existing cluster, making the BE container on the original single node part of the cluster.

[0026] Furthermore, the database cluster system based on Linux lightweight virtualization technology of the present invention also has the following features:

[0027] When performing expansion operations, the metadata management module retains the original single-machine address or abstracts it through VIP / LB for connection addresses, such as the address of Doris FE.

[0028] This invention also provides a database clustering method based on Linux lightweight virtualization technology, comprising the following steps:

[0029] Step 1: Configure the physical layer using standard x86 server hardware;

[0030] Step 2: Configure the operating system layer, using a Linux operating system that supports cgroups and LXC;

[0031] Step 3: Configure the virtualization layer and use the cgroups feature of the Linux kernel to create independent resource control groups for each application or its critical components to precisely control CPU time slices, memory usage and reclamation policies, block device I / O bandwidth / weight, and network bandwidth / priority;

[0032] Step 4: Configure the application layer to run key big data application components, including the Apache Doris database service FE / BE node, computing engine, and web services, in separate LXC containers.

[0033] Furthermore, the database clustering method based on Linux lightweight virtualization technology of the present invention also has the following characteristics:

[0034] In step four, if the initial configuration is in standalone mode, all necessary components, including Doris FE + Doris BE + default applications, run in their respective independent containers on the same physical node.

[0035] Furthermore, the database clustering method based on Linux lightweight virtualization technology of the present invention also has the following characteristics: it also has,

[0036] Step 5, Expansion Steps:

[0037] The metadata management module in the application layer is responsible for parsing and executing the operation of recording the address and port information of the new BE into the cluster's metadata, based on preset policies or administrator commands.

[0038] When the metadata management module receives an instruction to scale up to cluster mode, it quickly deploys a new container on a new physical node via LXC, containing Doris BE and other components. It then calls the Doris management interface to dynamically add the new BE node to the existing cluster, and the BE container on the original single node becomes part of the cluster, thus completing the scaling operation.

[0039] Beneficial effects of the invention: This invention enables efficient utilization of hardware resources, strong isolation between applications to ensure stability, rapid elastic scaling of resources, and smooth and seamless evolution from stand-alone machines to distributed clusters. Ultimately, it significantly reduces the hardware investment cost and operational complexity of enterprise big data implementation and makes the hardware investment curve smoother. Attached Figure Description

[0040] Figure 1 This is an architecture diagram of a database cluster system based on Linux lightweight virtualization technology. Detailed Implementation

[0041] The specific embodiments of the present invention will be described below with reference to the accompanying drawings.

[0042] First, let's explain the meaning of the terms used in the implementation: cgroups - control groups.

[0043] LXC is an abbreviation for Linux Container.

[0044] Namespaces — Namespaces.

[0045] BE is an abbreviation for Backend node.

[0046] FE is an abbreviation for Frontend node.

[0047] like Figure 1 As shown, a database cluster system based on Linux lightweight virtualization technology includes:

[0048] Physical layer 1: Standard x86 server hardware.

[0049] Operating System Layer 2: Unify the installation of Linux distributions that support cgroups and LXC, such as Ubuntu and CentOS.

[0050] Virtualization Layer 3: Utilizes the Linux kernel's cgroups feature to create independent resource control groups for each application or its critical components. This allows for precise control over CPU time slices, memory usage and reclamation policies, block device I / O, bandwidth / weight, network bandwidth / priority, etc.

[0051] Application Layer 4: Key big data application components, such as the FE / BE nodes of the Apache Doris database service, the computing engine, and web services, run in independent LXC containers.

[0052] LXC is an abbreviation for Linux Containers. It is a lightweight virtualization technology at the operating system level that achieves process and resource isolation by sharing the same operating system kernel. It primarily relies on Linux kernel cgroups for resource management, such as CPU and memory limits, and namespaces for isolation, such as processes, networks, and file systems. Namespaces, such as PID, NET, MNT, IPC, and UTS, achieve isolation between containers and between containers and the host. Each container has its own view of processes, network, and file systems. The cgroups control group is used to limit and allocate hardware resources for each container, such as CPU, memory, and disk I / O, ensuring they do not interfere with each other.

[0053] Ensure resource isolation between applications. Integrate and configure databases such as Apache Doris. Doris supports dynamically adding / removing Backend (BE) nodes and Frontend (FE) nodes online.

[0054] FE and BE are two core components in a distributed architecture. FE is the "brain" of the cluster, responsible for management, scheduling and coordination, metadata management, query parsing and planning, cluster management, user request access to the high-availability architecture Leader-Follower-Observer, and full-memory metadata storage.

[0055] The BE (Branch Execution Environment) is the actual operator of the cluster, responsible for data storage and management, query plan execution, data replication and sharded columnar storage, data sharding tablets, multi-replica mechanisms, and a vectorized execution engine. Both FE (Functional Execution Environment) and BE can be horizontally scaled.

[0056] Adding BE nodes can improve storage and computing capabilities; adding FE Observer nodes can enhance query access capabilities.

[0057] New nodes are quickly deployed and added to the cluster as LXC containers. It includes a configuration management module: storing and managing system deployment modes—single machine / cluster, node information, application configuration, etc. It also includes a monitoring and scheduling module, monitoring host machine and container resource usage, such as CPU, Mem, IO, Net, and database cluster load and status. It triggers scaling up / down operations based on preset policies or administrator commands.

[0058] Regarding the seamless switching control logic, in standalone mode: all necessary components, such as Doris FE + Doris BE + other applications, run in their own independent containers on the same physical node.

[0059] When switching to cluster mode is required: The system automatically, or with administrator confirmation, quickly deploys a new container—containing Doris BE or other components—on a new physical node via LXC, and dynamically adds the new BE node to the existing cluster by calling the Doris management interface. The BE container on the original single node automatically becomes part of the cluster.

[0060] Application layer configurations do not need to be changed. Connection addresses, such as those for Doris FE, are typically retained as the original standalone addresses or abstracted through VIP / LB. The entire scaling process is transparent to upper-layer applications or causes minimal disruption.

[0061] The reduction process is the reverse of the steps described above.

[0062] Pointing to an existing FE Leader: New nodes, especially BEs and Follower FEs, need to specify at least one existing FE node, usually the address of the Leader, through the FE_SERVERS environment variable or configuration file.

[0063] Here's the specific procedure: To ensure that newly deployed BE containers and existing single-node BE containers form a cluster, it's crucial to ensure they are correctly registered to the same Doris cluster's FE metadata. Doris cluster membership is managed by the FE. In a single-node deployment, the BE is typically on the same node as the FE and is registered to the FE using the command `ALTER SYSTEM ADD BACKEND "existing_be_host:heartbeat_service_port";` or `ALTER SYSTEM ADD BACKEND "192.168.1.100:9050";`. At this point, it's already part of the cluster, even if it's the only BE in the cluster. After deploying the new BE container on the new physical node via LXC, the Doris management interface is called, primarily through SQL commands executed by the FE, to add the new BE node's information to the cluster's metadata. Connect to the FE via a MySQL client and execute the command to add a BE: `ALTER SYSTEM ADD BACKEND "new_be_host:heartbeat_service_port";` where `new_be_host` is the IP address or hostname of the new BE container, and `heartbeat_service_port` is the BE's heartbeat port, which defaults to 9050. After executing this command, the FE will begin to include the new BE in its management scope and gradually perform data balancing. The Doris FE and BE maintain communication through a heartbeat mechanism. The new BE node needs to "inform" the FE via the above SQL command; this is for security and stability reasons, preventing unauthorized nodes from joining the cluster.

[0064] The above process can be automated using scripts to automatically add new BEs: after LXC successfully creates and starts a new Doris BE container, the script can automatically execute the `ALTER SYSTEM ADD BACKEND` command to register the new BE. This can be achieved by integrating a MySQL client into the system, such as Ansible or SaltStack, to call SQL commands from the FE.

[0065] Administrator confirmation mechanism: An "approval" or "confirmation" step can be added to the automated process. For example, after the automated platform triggers an expansion task, it notifies the administrator for review. Only after the administrator approves can the system execute the command to add the BE (Body Environment).

[0066] Status check: After the new BE container starts and executes the registration command, the automated process should check the joining status, for example by querying SHOW PROC ' / backends'; to confirm whether the Alive status of the new BE is true.

[0067] The existing BE remains unchanged: During the scaling process, no special operations are required on the existing BE container on the single node. It is already running as an existing node in the cluster, and the scaling operation simply adds a container running parallel to it.

[0068] Based on the above system, this embodiment provides a database clustering method using Linux lightweight virtualization technology, including the following steps:

[0069] Step 1: Configure the physical layer using standard x86 server hardware;

[0070] Step 2: Configure the operating system layer, using a Linux operating system that supports cgroups and LXC;

[0071] Step 3: Configure the virtualization layer and use the cgroups feature of the Linux kernel to create independent resource control groups for each application or its critical components to precisely control CPU time slices, memory usage and reclamation policies, block device I / O bandwidth / weight, and network bandwidth / priority;

[0072] Step 4: Configure the application layer to run key big data application components, including the Apache Doris database service FE / BE node, computing engine, and web services, in separate LXC containers.

[0073] In step four, if the initial configuration is in standalone mode, all necessary components, including Doris FE + Doris BE + default applications, run in their respective independent containers on the same physical node.

[0074] When expansion is required, proceed to step five, the expansion steps:

[0075] The metadata management module in the application layer is responsible for parsing and executing the operation of recording the address and port information of the new BE into the cluster's metadata, based on preset policies or administrator commands.

[0076] When the metadata management module receives an instruction to scale up to cluster mode, it quickly deploys a new container on a new physical node via LXC, containing Doris BE and other components. It then calls the Doris management interface to dynamically add the new BE node to the existing cluster, and the BE container on the original single node becomes part of the cluster, thus completing the scaling operation.

[0077] Similarly, if in step four the initial configuration is in cluster mode, and you want to scale down, you can reverse the operation and delete the container.

[0078] The advantages and disadvantages of the prior art and the present invention are compared in the following table.

[0079] Prior art categories Main disadvantages Advantages and beneficial effects of the invention Single-machine deployment (bare metal) 1. No resource isolation: resource conflicts are serious when multiple applications coexist, leading to unstable applications and mutual interference. 2. Low resource utilization: it is difficult to finely allocate and reuse resources. 3. Poor scalability: only vertical expansion is possible, limited by the single machine. 1. Strong resource isolation: cgroups (control groups) are used to achieve fine isolation and limitation of resources such as CPU, memory, disk IO, and network bandwidth, ensuring that applications do not interfere with each other and significantly improving system stability. 2. Resource efficient reuse: the lightweight nature allows multiple isolated environments to share the same kernel, with high resource reuse rate. 3. Lightweight and fast startup: LXC containers start quickly with minimal resource overhead. Virtual machine technology (e.g. VMware, KVM) 1. Large resource overhead: each VM runs an independent OS kernel, consuming a large amount of memory and CPU. 2. Performance loss: hardware virtualization layer brings additional performance overhead (especially I / O). 3. Slow elasticity: creating, starting, and cloning VMs is relatively time-consuming, and scaling is not flexible and efficient. 1. Extremely lightweight: based on containerization technology, there is no independent OS kernel overhead, and the resource utilization rate is much higher than that of VM. 2. Close to native performance: directly use the host kernel, with very low virtualization loss, especially excellent I / O performance. 3. Second-level elasticity: containers start, stop, and copy extremely fast, supporting fast horizontal expansion (Scale-Out) and contraction (Scale-In). Hyper-converged (HCI), Kubernetes (K8s) 1. Complex architecture: involves multiple technology stacks (virtualization layer, distributed storage layer, orchestration layer), with high deployment, configuration, and operation complexity. 2. Cumulative performance loss: multiple layers of abstraction result in cumulative performance loss, which is not friendly to high-performance demand scenarios. 3. High resource threshold: usually requires more nodes and management resources to take advantage of, and small-scale deployment has low cost-effectiveness. 1. Simple architecture: relies on Linux native lightweight virtualization (cgroups / namespaces) and database dynamic expansion capabilities, with simple and clear architecture. 2. Efficient and low consumption: avoids complex intermediate layers, with small overall performance loss, especially suitable for real-time analysis scenarios sensitive to performance. 3. Low threshold and smooth evolution: supports starting from a single node with low cost, and seamlessly transitions to a distributed cluster mode as the business grows, with hardware investment increasing smoothly as the business grows. Traditional big data cluster deployment 1. Static configuration: cluster size adjustment usually requires downtime or complex operations. 2. Poor elasticity: slow response to load changes. 3. Single point bottleneck: single-machine deployment has a performance upper limit and single-point failure risk. 1. Database-driven dynamic scaling: Take advantage of the dynamic scaling / shrinking features of databases such as Apache Doris. The cluster size can be automatically or manually adjusted according to the load, without the need for complex configuration or downtime.2. Seamless architecture switching: The system design supports the same set of applications and configurations, which can efficiently run in the "all-in-one" mode of a single node (LXC container) and can seamlessly and transparently switch to the distributed cluster mode by dynamically adding nodes (also deployed in LXC containers) and using the database scaling capability when the load increases. This greatly simplifies the enterprise architecture evolution path.

Claims

1. A Linux Container-based database cluster system, characterized in that, Comprise: Physical layer: with standard x86 server hardware; Operating system layer: with Linux operating system supporting cgroups and LXC functions; Virtualization layer: based on cgroups functions of Linux kernel, create independent resource control groups for each application or its key components, accurately control CPU time slice, memory usage and recycling strategy, block device I / O bandwidth and weight, and network bandwidth and priority; Application layer: key big data application components, including FE / BE nodes of database service Apache Doris, computing engine and web service, running in independent LXC containers respectively.

2. The database cluster system based on Linux lightweight virtualization technology according to claim 1, characterized in that: It contains a configuration management module: storing and managing system deployment mode: single machine or cluster, node information and application configuration.

3. The database cluster system based on Linux lightweight virtualization technology according to claim 2, characterized in that: In single machine mode: Doris FE, Doris BE and preset applications all run in their own independent containers on the same physical node.

4. The database cluster system based on Linux lightweight virtualization technology according to claim 1, characterized in that: It contains a monitoring and scheduling module, which monitors the resource usage of host and container, including CPU, Mem, IO, Net, database cluster load and state.

5. The database cluster system based on Linux lightweight virtualization technology according to claim 2, characterized in that: It has a metadata management module, based on preset strategy or administrator instruction, responsible for parsing and executing the operation of recording the address and port information of new BE to the metadata of the cluster, completing the expansion operation.

6. The database cluster system based on Linux lightweight virtualization technology according to claim 5, characterized in that: The metadata management module is used to quickly deploy a new container containing Doris BE on a new physical node through LXC when receiving the instruction to expand to cluster mode, and call the Doris management interface to dynamically add the new BE node to the existing cluster, so that the BE container on the original single node becomes part of the cluster.

7. The database cluster system based on Linux lightweight virtualization technology according to claim 6, characterized in that: The metadata management module keeps the original single machine address or abstracts through VIP / LB when performing expansion operation for connection address such as Doris FE address.

8. A database cluster method based on Linux lightweight virtualization technology, comprising the steps of: Step one: configure the physical layer, adopt standard x86 server hardware; Step two: configure the operating system layer, adopt Linux operating system supporting cgroups and LXC; Step 3: Configure the virtualization layer, use the cgroups function of the Linux kernel to create independent resource control groups for each application or its key components, and accurately control CPU time slices, memory usage and recovery strategies, block device I / O bandwidth / weight, and network bandwidth / priority. Step 4: Configure the application layer, run key big data application components, including FE / BE nodes of the database service Apache Doris, computing engines, and web services, in separate LXC containers.

9. The database cluster method based on the Linux lightweight virtualization technology according to claim 8, characterized in that: In step 4, if the initial configuration is single-machine mode, all necessary components, including Doris FE + Doris BE + preset applications, run in separate containers on the same physical node.

10. The Linux-based lightweight virtualization technology based database cluster method as claimed in claim 8, wherein, Also has, Step 5: Expansion step: The metadata management module in the application layer is responsible for parsing and executing the operation of recording the address and port information of the new BE to the metadata of the cluster based on the preset strategy or administrator instructions; When receiving the instruction to expand to the cluster mode, the metadata management module quickly deploys a new container containing Doris BE and other components on the new physical node through LXC, calls the Doris management interface to dynamically add the new BE node to the existing cluster, and the BE container on the original single node becomes part of the cluster, completing the expansion operation.

Images