OpenGauss cluster deployment method based on cloud platform

Abstract

The invention provides an openGauss cluster deployment method based on a cloud platform, belongs to the technical field of cloud computing and databases, and realizes a full-automatic deployment process from virtual machine creation to cluster configuration through the cloud platform, a management back end and a cluster deployment script. The method comprises the following specific steps: virtual machine creation: a cloud platform creates a virtual machine through a virtualization technology, and verifies network connectivity; and environment detection: the management back end performs environment detection on the virtual machine to generate a cluster configuration file. And cluster mutual trust: configuring virtual machine mutual trust by a cluster script to ensure mutual access among nodes. The virtual machine configuration comprises environment variable setting, database tool installation, database software package distribution, cron service configuration and the like. System adjustment and optimization: system parameter optimization is carried out for an operating system and a server architecture. And cluster configuration: configuring database parameters, starting a database service, and ensuring that the cluster state is normal. According to the method, the deployment difficulty can be reduced, the manual operation is reduced, the deployment efficiency and the cluster performance are improved, and meanwhile, the method has relatively good compatibility, expandability and maintainability.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing and database technology, and in particular to an openGauss cluster deployment method based on a cloud platform. Background Art

[0002] With the rapid development of cloud computing technology, more and more enterprises and organizations choose to migrate their data and applications to cloud platforms. openGauss is a high-performance, high-availability, and high-security open source database system suitable for enterprise-level applications of all sizes. However, the existing openGauss cluster deployment methods on cloud platforms have problems such as complex deployment, prone to errors, and difficulty in adapting to different cloud platform environments. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides an openGauss cluster deployment method based on a cloud platform, which can simplify the deployment process and has good scalability.

[0004] The technical solution of the present invention is:

[0005] A cloud platform-based openGauss cluster deployment method includes the following steps:

[0006] The cloud platform uses virtualization technology to open up virtual machines with the same number of cluster nodes from the tenant side for deploying the openGauss cluster. After opening the virtual machines, the network connectivity of the virtual machines is verified to ensure that the management backend is connected to the virtual machine network, and then detailed information of each node virtual machine is returned.

[0007] The management backend obtains the virtual machine information, performs environmental detection on each virtual machine in the deployment cluster, and generates a cluster configuration file cluster.xml. After passing the environmental detection, it triggers the cluster deployment script copied to the master node.

[0008] The cluster script reads the cluster configuration file to establish mutual trust between cluster virtual machines, so that the master node can access other nodes.

[0009] The cluster script configures the virtual machine, modifies environment variables and environment files, checks and configures the hosts file, installs tools and configures tool paths, and configures cron services.

[0010] The cluster script performs system tuning, optimizes system parameters, sets operating system parameters, and performs targeted optimization for the trusted ARM environment.

[0011] The cluster script configures the cluster, configures database parameters, starts the openGauss database service on each node, and after checking that the cluster status information is normal, configures the access IP to complete the cluster configuration and application.

[0012] Furthermore,

[0013] The cloud platform returns detailed information about the tenant-side cluster virtual machine, including:

[0014] Newly opened management IP information of each node virtual machine;

[0015] The operating system image version of each virtual machine;

[0016] Customizable ssh service port, login user and password information;

[0017] Virtual machine specification information, including CPU, memory, and disk information.

[0018] Furthermore,

[0019] Manage cluster configuration files generated by the backend, including:

[0020] According to the detailed information of the tenant-side cluster virtual machine returned by the cloud platform, the management backend selects a node as the master node and configures it as the master node information, including: master node cluster information and cluster cm configuration information.

[0021] Configure the information of each node in the database cluster, including: cluster name, AZ region where the cluster node is located, node name, node IP, node operating system, database user name, database password, installation directory, log directory, and tool directory information.

[0022] Furthermore,

[0023] The management backend performs environmental testing on each virtual machine in the deployment cluster, including the following:

[0024] Check whether the virtual machine operating system version meets the cluster version requirements for installing openGauss. If not, an error log is thrown to stop subsequent operations.

[0025] Check the system dependency environment information and check whether the expect installation package version information meets the version requirements for building the openGauss cluster. If not, install or upgrade it.

[0026] Check the environment permission settings. The installation directory / opt / software / openGauss needs to be granted permission 755 for the user. If there is any inconsistent permissions, correct them.

[0027] Checks whether the instance directory conflicts with the application directory and throws an error if there is a directory conflict.

[0028] Furthermore,

[0029] The cluster script includes the following contents to establish mutual trust between cluster virtual machines:

[0030] Obtain the ssh user name, password, and port information of the master node, standby node, and cascade node from the cluster configuration file.

[0031] Configure mutual trust between the master node and the standby node so that they can trust each other and access each other through SSH through the hostname.

[0032] Configure mutual trust between the standby node and the cascade node so that the standby node and the cascade node can access each other through SSH through the hostname.

[0033] Furthermore,

[0034] The cluster script configuration virtual machine includes the following:

[0035] Pull the cluster software package of the corresponding operating system version from the software source repository to the installation directory of the master node, and decompress it to the installation directory.

[0036] Set environment variables and environment files, delete the HOST_IP variable in the / etc / profile file, and configure the environment check script check_envfile file.

[0037] Distribute server packages and distribute the pulled cluster software packages to other standby nodes or cascade nodes.

[0038] Configure cron services, including environment check tasks and heartbeat detection and disaster recovery tasks.

[0039] Check and configure the hosts file, write the IP and hostname information of each node into the / etc / hosts file to ensure that remote ssh operations can be performed through the hostname of each node later.

[0040] Furthermore,

[0041] The cluster script for cluster configuration includes the following:

[0042] The cluster form and configuration strategy are determined according to the cluster configuration file. The cluster must ensure that there can be only one master node and several backup nodes. When there are backup nodes, there can be cascade nodes. The master node cluster script connects to each node through mutual trust for configuration.

[0043] Configure remote access to the master node, that is, the node currently executing the script. Modify the openGauss configuration file, including configuring remote access information in pg_hba.conf, configuring password policies and listen_addresses parameters in postgresql.conf. Ensure that pg_hba.conf allows access from the standby node.

[0044] For the standby node, configure it to connect to the primary node.

[0045] For a cluster with cascaded nodes, configure the standby node to allow access to the cascaded nodes and configure synchronization of the cascaded nodes to the standby node.

[0046] After configuring cluster-related configuration files for all nodes and starting the openGauss services of all nodes, the management backend executes gs_om-t status on the master node to obtain cluster information. After ensuring that the cluster status is normal, configure the access IP address of the master node to complete cluster configuration deployment.

[0047] Furthermore,

[0048] Set up the openGauss cluster deployment device, including:

[0049] The cloud platform virtualization module is used to virtualize the computing, storage, and network resources of several physical servers to obtain activated database instance virtual machines.

[0050] The management backend module is used to transmit virtual machine information and obtain the cluster deployment configuration file, which is used to manage and trigger the cluster deployment script. After the cluster status information is normal, the access IP is configured for the master node to obtain the openGauss cluster accessible to users.

[0051] The cluster building module is used to execute cluster mutual trust, virtual machine configuration, system tuning, and cluster configuration actions on each node of the cluster virtual machine, build a database cluster and maximize cluster performance.

[0052] The beneficial effects of the present invention are

[0053] 1. It can reduce the difficulty of deployment, reduce the complexity of manual operations, improve deployment efficiency, quickly complete deployment, maximize automation, improve service response capabilities, and reduce costs, thereby improving the installation and deployment efficiency of the openGauss cluster and achieving high feasibility;

[0054] 2. The present invention realizes the automatic deployment of various cluster forms of openGauss. During the deployment, it can also complete the tuning of the database cluster according to different operating systems and cluster scales, and perform tuning for the ARM environment to improve cluster performance;

[0055] 3. The present invention can adapt to different cloud platform environments and has good compatibility and adaptability;

[0056] 4. The present invention also has good scalability and maintainability, and is convenient for subsequent upgrading and expansion. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1It is a schematic diagram of the workflow of the present invention;

[0058] Figure 2 This is a flowchart of the environment detection process in the openGauss cluster installation and deployment method;

[0059] Figure 3 This is a flowchart of cluster configuration in the openGauss cluster installation and deployment method. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0061] The following steps are involved:

[0062] The cloud platform uses virtualization technology to open virtual machines of the number of cluster nodes from the tenant side for deploying the openGauss cluster. After opening the virtual machines, the network connectivity of the virtual machines is verified to ensure that the management backend is connected to the virtual machine network, and then the detailed information of each node virtual machine is returned;

[0063] The management backend obtains the virtual machine information, performs environmental detection on each virtual machine in the deployment cluster, and generates a cluster configuration file cluster.xml. After passing the environmental detection, it triggers the cluster deployment script copied to the master node;

[0064] The cluster script reads the cluster configuration file to establish mutual trust between cluster virtual machines, so that the master node can access other nodes;

[0065] The cluster script configures the virtual machine, modifies environment variables and environment files, checks and configures the hosts file, installs tools and configures tool paths, and configures cron services;

[0066] Cluster scripts perform system tuning, optimize system parameters, set operating system parameters, and perform targeted optimization for the Xinchuang ARM environment;

[0067] The cluster script configures the cluster, configures database parameters, starts the openGauss database service on each node, and after checking that the cluster status information is normal, configures the access IP to complete the cluster configuration and application.

[0068] The cloud platform returns detailed information about the tenant-side cluster virtual machine, including:

[0069] Newly opened management IP information of each node virtual machine;

[0070] The operating system image version of each virtual machine;

[0071] Customizable ssh service port, login user and password information;

[0072] Virtual machine specification information, including CPU, memory, and disk information;

[0073] Manage cluster configuration files generated by the backend, including:

[0074] According to the detailed information of the tenant-side cluster virtual machine returned by the cloud platform, the management backend selects a node as the master node and configures it as the master node information, including: master node cluster information and cluster cm configuration information;

[0075] Configure the information of each node in the database cluster (including the master node), including: cluster name, AZ region where the cluster node is located, node name, node IP, node operating system, database user name, database password, installation directory, log directory, and tool directory information.

[0076] Preferably, the management backend performs environmental detection on each virtual machine of the deployment cluster including the following:

[0077] Check whether the virtual machine operating system version meets the cluster version requirements for installing openGauss. If not, an error log is thrown to stop subsequent operations.

[0078] Detect system dependency environment information and check whether the expect installation package version information meets the version requirements for building the openGauss cluster. If not, install or upgrade.

[0079] Check the environment permission settings. The installation directory / opt / software / openGauss needs to be granted permission 755 for the user. If there is any inconsistent permissions, correct it.

[0080] Checks whether the instance directory conflicts with the application directory and throws an error if there is a directory conflict.

[0081] The cluster script includes the following contents to establish mutual trust between cluster virtual machines:

[0082] Get the ssh username, password, and port information of the master node, standby node, and cascade node from the cluster configuration file;

[0083] Configure mutual trust between the master node and the standby node so that they can trust each other and access each other through SSH through hostname;

[0084] Configure mutual trust between the standby node and the cascade node so that the standby node and the cascade node can access each other through SSH through the hostname.

[0085] The cluster script configuration virtual machine includes the following:

[0086] Pull the cluster software package of the corresponding operating system version from the software source warehouse to the installation directory of the master node, and decompress it to the installation directory;

[0087] Set environment variables and environment files, delete the HOST_IP variable in the / etc / profile file, and configure the environment check script check_envfile file;

[0088] Distribute server packages and distribute the pulled cluster software packages to other standby nodes or cascade nodes;

[0089] Configure cron services, including environment check tasks and heartbeat detection and disaster recovery tasks;

[0090] Check and configure the hosts file, write the IP and hostname information of each node into the / etc / hosts file to ensure that remote ssh operations can be performed through the hostname of each node later.

[0091] The cluster script for cluster configuration includes the following:

[0092] Determine the cluster form and configuration strategy based on the cluster configuration file. The cluster must ensure that there can be only one master node and multiple standby nodes. When there are standby nodes, there can be cascade nodes. The master node cluster script connects to each node through mutual trust for configuration.

[0093] Configure remote access to the master node, i.e. the node currently executing the script. Modify the openGauss configuration file, including configuring remote access information in pg_hba.conf, and configuring password policies and listen_addresses and other parameters in postgresql.conf. Ensure that the pg_hba.conf configuration allows access from the standby node.

[0094] For the standby node, configure it to connect to the master node;

[0095] For a cluster with cascaded nodes, configure the standby node to allow access to the cascaded nodes and configure synchronization of the cascaded nodes to the standby node;

[0096] After configuring cluster-related configuration files for all nodes and starting the openGauss services of all nodes, the management backend executes gs_om-t status on the master node to obtain cluster information. After ensuring that the cluster status is normal, configure the access IP address of the master node to complete cluster configuration deployment.

[0097] The openGauss cluster deployment device comprises:

[0098] The cloud platform virtualization module is used to virtualize the computing, storage, and network resources of multiple physical servers to obtain the activated database instance virtual machines;

[0099] The management backend module is used to transmit virtual machine information and obtain cluster deployment configuration files, which are used to manage and trigger cluster deployment scripts. After the cluster status information is normal, the access IP address of the master node is configured to obtain an openGauss cluster accessible to users.

[0100] The cluster building module is used to execute cluster mutual trust, virtual machine configuration, system tuning, and cluster configuration actions on each node of the cluster virtual machine, build a database cluster and maximize cluster performance.

[0101] The present invention also provides an electronic device, comprising:

[0102] Memory for storing computer programs;

[0103] A processor is used to implement the steps of the above-mentioned fault handling method when executing the computer program.

[0104] The present invention also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned fault handling method are implemented.

[0105] The present invention can reduce the difficulty of deployment, reduce the complexity of manual operations, improve deployment efficiency, quickly complete deployment, complete it automatically to the greatest extent, improve service responsiveness, and reduce costs, thereby improving the installation and deployment efficiency of the openGauss cluster and having high feasibility. It can realize the deployment of various cluster forms of openGauss, and at the same time can complete the tuning of the database cluster according to different operating systems and cluster sizes, and can perform tuning for the trusted ARM environment to improve cluster performance.

[0106] The above description is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A cloud platform-based openGauss cluster deployment method, characterized in that: The following steps are involved: The cloud platform uses virtualization technology to open virtual machines of the number of cluster nodes from the tenant side for deploying the openGauss cluster. After opening the virtual machines, the network connectivity of the virtual machines is verified to ensure that the management backend is connected to the virtual machine network, and then the detailed information of each node virtual machine is returned; The management backend obtains the virtual machine information, performs environmental detection on each virtual machine in the deployment cluster, and generates a cluster configuration file cluster.xml. After passing the environmental detection, it triggers the cluster deployment script copied to the master node; The cluster script reads the cluster configuration file to establish mutual trust between cluster virtual machines, so that the master node can access other nodes; The cluster script configures the virtual machine, modifies environment variables and environment files, checks and configures the hosts file, installs tools and configures tool paths, and configures cron services; Cluster scripts perform system tuning, optimize system parameters, set operating system parameters, and perform targeted optimization for the Xinchuang ARM environment; The cluster script configures the cluster, configures database parameters, starts the openGauss database service on each node, and after checking that the cluster status information is normal, configures the access IP to complete the cluster configuration and application.

2. The method according to claim 1, characterized in that Detailed information about the tenant-side cluster VM returned by the cloud platform, including: Newly opened management IP information of each node virtual machine; The operating system image version of each virtual machine; Customizable ssh service port, login user and password information; Virtual machine specification information, including CPU, memory, and disk information.

3. The method according to claim 1, characterized in that Manage cluster configuration files generated by the backend, including: According to the detailed information of the tenant-side cluster virtual machine returned by the cloud platform, the management backend selects a node as the master node and configures it as the master node information, including: master node cluster information and cluster cm configuration information.

4. The method according to claim 3, characterized in that Configure the information of each node in the database cluster, including: cluster name, AZ region where the cluster node is located, node name, node IP, node operating system, database user name, database password, installation directory, log directory, and tool directory information.

5. The method according to claim 1, characterized in that The management backend performs environmental testing on each virtual machine in the deployment cluster, including the following: Check whether the virtual machine operating system version meets the cluster version requirements for installing openGauss. If not, an error log is thrown to stop subsequent operations. Detect system dependency environment information and check whether the expect installation package version information meets the version requirements for building the openGauss cluster. If not, install or upgrade. Check the environment permission settings. The installation directory / opt / software / openGauss needs to be granted permission 755 for the user. If there is any inconsistent permissions, correct it. Checks whether the instance directory conflicts with the application directory and throws an error if there is a directory conflict.

6. The method according to claim 1, characterized in that The cluster script includes the following contents to establish mutual trust between cluster virtual machines: Get the ssh username, password, and port information of the master node, standby node, and cascade node from the cluster configuration file; Configure mutual trust between the master node and the standby node so that they can trust each other and access each other through SSH through hostname; Configure mutual trust between the standby node and the cascade node so that the standby node and the cascade node can access each other through SSH through the hostname.

7. The method according to claim 1, characterized in that The cluster script configuration virtual machine includes the following: Pull the cluster software package of the corresponding operating system version from the software source warehouse to the installation directory of the master node, and decompress it to the installation directory; Set environment variables and environment files, delete the HOST_IP variable in the / etc / profile file, and configure the environment check script check_envfile file; Distribute server packages and distribute the pulled cluster software packages to other standby nodes or cascade nodes; Configure cron services, including environment check tasks and heartbeat detection and disaster recovery tasks; Check and configure the hosts file, write the IP and hostname information of each node into the / etc / hosts file to ensure that remote ssh operations can be performed through the hostname of each node later.

8. The method according to claim 1, characterized in that The cluster script for cluster configuration includes the following: Determine the cluster form and configuration strategy based on the cluster configuration file. The cluster must ensure that there can be only one master node and several backup nodes. When there are backup nodes, there can be cascade nodes. The master node cluster script connects to each node through mutual trust for configuration. Configure remote access to the master node, i.e. the node currently executing the script. Modify the openGauss configuration file, including configuring remote access information in pg_hba.conf, configuring password policy and listen_addresses parameter configuration in postgresql.conf. Ensure that the pg_hba.conf configuration allows access from the standby node. For the standby node, configure it to connect to the master node; For a cluster with cascaded nodes, configure the standby node to allow access to the cascaded nodes and configure synchronization of the cascaded nodes to the standby node; After configuring cluster-related configuration files for all nodes and starting the openGauss services of all nodes, the management backend executes gs_om-t status on the master node to obtain cluster information. After ensuring that the cluster status is normal, configure the access IP address of the master node to complete cluster configuration deployment.

9. The method according to claim 1, characterized in that: Set up the openGauss cluster deployment device, including: The cloud platform virtualization module is used to virtualize the computing, storage, and network resources of several physical servers to obtain the activated database instance virtual machines; The management backend module is used to transmit virtual machine information and obtain cluster deployment configuration files, which are used to manage and trigger cluster deployment scripts. After the cluster status information is normal, the access IP address of the master node is configured to obtain an openGauss cluster accessible to users. The cluster building module is used to execute cluster mutual trust, virtual machine configuration, system tuning, and cluster configuration actions on each node of the cluster virtual machine, build a database cluster and maximize cluster performance.

Images