A method, apparatus and medium for installation and deployment of a cluster
By automating cluster installation and deployment methods and utilizing remote connections and batch processing of script files, the ClickHouse cluster can be installed and configured efficiently, solving the problems of tediousness and omissions in traditional methods and improving installation efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INSPUR SUZHOU INTELLIGENT TECH CO LTD
- Filing Date
- 2022-06-30
- Publication Date
- 2026-07-24
Smart Images
Figure CN115167878B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of big data storage technology, and in particular to a method, apparatus and medium for installing and deploying a cluster. Background Technology
[0002] In recent years, with the development of big data business demands and the growth in data volume, the query speed of traditional databases can no longer meet business needs. More and more enterprises are choosing to use ClickHouse cluster databases for distributed storage of large amounts of data. ClickHouse uses ZooKeeper to ensure data consistency when implementing distributed storage, mainly for synchronizing replica table data (ReplicatedMergeTree engine) and operating distributed tables.
[0003] When using ClickHouse and ZooKeeper, deployment and installation issues arise. Many existing big data clusters are very large, with numerous database servers. Traditional ClickHouse cluster deployment methods require uploading installation files to each node and configuring them in each node's ` / etc / metrika.xml` file. This current method involves manual deployment and installation node by node, which is not only tedious and complex but also indirectly increases time costs and may even lead to node chaos due to omissions.
[0004] Therefore, finding an efficient method for installing and deploying clusters is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] The purpose of this invention is to provide a method, apparatus and medium for installing and deploying a cluster, which avoids configuring and operating each node individually, greatly simplifies the installation steps, saves installation and configuration time, and also solves the problem of missing and confused nodes in the existing system.
[0006] To address the aforementioned technical problems, this invention provides a method for installing and deploying a cluster, comprising:
[0007] Obtain multiple nodes in the cluster system, along with their corresponding IPs, script files, and program installation packages;
[0008] Store the script file and program installation package in the current node, where the current node is a single node;
[0009] The program installation package is copied to each node using the file copy command;
[0010] Establish remote connections based on the IP addresses of each node;
[0011] Install and configure each node using the installer;
[0012] The installation and deployment are achieved by batch starting each node through a processing program.
[0013] Preferably, each node is installed and configured using an installer, including:
[0014] Get the number of nodes and set the loop count based on the number;
[0015] Install the program's RPM package at the current node, and cycle through the installation of the current node as one installer of the RPM package.
[0016] Determine if the current installer's RPM package loop count exceeds the loop limit;
[0017] If the current installer's RPM package loop count has not exceeded the loop count, then obtain the next current node and return to the step where the installer's RPM package was installed at the current node;
[0018] If the current installer's RPM package loop count exceeds the loop count, return to the current node, install the ClickHouse package of the installer at the current node, modify the database configuration parameters of the current node, and treat the installation of the current node as one loop of installing the ClickHouse package;
[0019] Determine if the current number of times the ClickHouse package is installed exceeds the loop count;
[0020] If the current loop count for installing the ClickHouse package has not exceeded the loop count, then obtain the next current node and return to the steps of installing the ClickHouse package at the current node and modifying the database configuration parameters of the current node;
[0021] If the current loop count for installing the ClickHouse package exceeds the loop count, return to the current node, modify the corresponding cluster configuration file on the current node, and use the configuration of the current node as one configuration loop count;
[0022] Determine if the current configuration loop count exceeds the loop limit;
[0023] If the current configuration loop count has not exceeded the loop count, then obtain the next current node and return to the step of modifying the corresponding cluster configuration file at the current node;
[0024] If the current configuration loop count exceeds the loop count, return to the current node, install ZooKeeper and the corresponding configuration file on the current node, and use the current node's configuration as one ZooKeeper loop count;
[0025] Determine if the current ZooKeeper loop count exceeds the allowed number of loops;
[0026] If the current ZooKeeper loop count has not exceeded the loop count, then obtain the next current node and return to the step of installing ZooKeeper and the corresponding configuration file at the current node;
[0027] If the current ZooKeeper loop count exceeds the loop count, then the installation and configuration of each node are complete.
[0028] Preferably, after establishing a remote connection based on the IP of each node, and before installing and configuring each node using the installer, the method further includes:
[0029] Write the IP address corresponding to each node into the host of each node;
[0030] Perform the basic environment configuration to shut down each node.
[0031] Preferably, the basic environment configuration includes at least a firewall and an SELinux configuration package.
[0032] Preferably, before installing and configuring each node using the installer, the following steps are also included:
[0033] Obtain the server account and password for each node;
[0034] Enter the server account and password into the installation and deployment system of the current node.
[0035] Preferably, the nodes are started in batches through a processing program, including:
[0036] Start the ClickHouse and ZooKeeper programs by batch processing script files.
[0037] Preferably, it further includes:
[0038] When nodes are started in batches, a prompt message is output.
[0039] To address the aforementioned technical problems, the present invention also provides a cluster installation and deployment apparatus, comprising:
[0040] The acquisition module is used to acquire multiple nodes in the cluster system, their corresponding IPs, script files, and program installation packages.
[0041] The storage module is used to store script files and program installation packages to the current node, where there is only one current node;
[0042] The copy module is used to copy the program installation package to each node using file copy commands;
[0043] Establish a module to create remote connections based on the IP addresses of each node;
[0044] The configuration module is used to install and configure each node via the installer.
[0045] The startup module is used to start each node in batches through a processing program to achieve installation and deployment.
[0046] To address the aforementioned technical problems, the present invention also provides a cluster installation and deployment apparatus, comprising:
[0047] Memory, used to store computer programs;
[0048] A processor is used to execute computer programs to implement the steps of the cluster installation and deployment methods described above.
[0049] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the cluster installation and deployment method described above.
[0050] This invention provides a method for installing and deploying a cluster, including obtaining multiple nodes in the cluster system along with their corresponding IPs, script files, and program installation packages; storing the script files and program installation packages on a single current node; copying the program installation packages to each node using a file copy command; establishing remote connections based on the IPs of each node; installing and configuring each node using an installer; and batch starting each node using a processing program to achieve installation and deployment. This method, through an installer and processing program, enables unified installation, deployment, and configuration across multiple nodes using only a single node. It automatically uploads the program installation package via a file copy command, establishes connections between nodes via remote connections, automatically installs and modifies configurations, and then automatically starts the nodes. This avoids individual configuration and operation for each node, greatly simplifying the installation process, saving installation and configuration time, and also solving the problem of missing or disorganized nodes in existing systems.
[0051] In addition, the present invention also provides a cluster installation and deployment apparatus and medium, which have the same beneficial effects as the cluster installation and deployment method described above. Attached Figure Description
[0052] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0053] Figure 1 A flowchart illustrating a cluster installation and deployment method provided in an embodiment of the present invention;
[0054] Figure 2 A structural diagram of a cluster installation and deployment device provided in an embodiment of the present invention;
[0055] Figure 3 This is a structural diagram of another cluster installation and deployment device provided in an embodiment of the present invention. Detailed Implementation
[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.
[0057] The core of this invention is to provide a method, apparatus and medium for installing and deploying a cluster, which avoids configuring and operating each node individually, greatly simplifies the installation steps, saves installation and configuration time, and also solves the problem of missing and confused nodes in the existing system.
[0058] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0059] It should be noted that the cluster installation and deployment method provided in this invention is applicable not only to distributed cluster databases ClickHouse and ZooKeeper, but also to other distributed cluster databases, without specific limitations. ClickHouse is a column-oriented database management system (DBMS) using a massively parallel processing (MMP) architecture for online analytical processing (OLAP), capable of generating real-time analytical data reports using Structured Query Language (SQL). ClickHouse supports user behavior analysis, stream and batch processing, linear scalability, and reliability assurance, natively supporting sharding and replication. ClickHouse does not utilize the Hadoop ecosystem, instead employing LocalAttached Storage for storage.
[0060] ZooKeeper is a distributed, open-source distributed application coordination service and a crucial component of Hadoop and HBase. It's software that provides consistency services for distributed applications, offering features such as configuration maintenance, domain name service, distributed synchronization, and group services. ZooKeeper's goal is to encapsulate complex and error-prone critical services, providing users with simple, easy-to-use interfaces and a high-performance, stable system.
[0061] Figure 1 A flowchart illustrating a cluster installation and deployment method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes:
[0062] S11: Obtain multiple nodes in the cluster system, their corresponding IPs, script files, and program installation packages;
[0063] S12: Store the script file and program installation package in the current node, where the current node is one;
[0064] S13: Copy the program installation package to each node using the file copy command;
[0065] S14: Establish remote connections based on the IP addresses of each node;
[0066] S15: Install and configure each node using the installer;
[0067] S16: Batch startup of each node through a processing program to achieve installation and deployment.
[0068] Specifically, the process involves obtaining multiple nodes in the cluster system and their corresponding Internet Protocol Addresses (IPs), script files, and program installation packages. The methods for obtaining the nodes and their corresponding IPs differ from those for obtaining the script files and program installation packages. The nodes and their corresponding IPs are obtained through user input. For the script files and program installation packages, a new folder is created in the root directory of one of the server nodes to be installed. The compressed file containing the script files and program installation packages is then uploaded to this directory and decompressed to obtain the script files and program installation packages.
[0069] The folder in the root directory has no special requirements; it can be any new folder or any folder within the root directory, such as the cfnet folder. The key is to store the script files and program installation packages on the current node server to be installed. This embodiment corresponds to storing the script files and program installation packages on the current node in step S12 above. It should be noted that the current node can be any node server to be installed, but there can only be one. The installation and deployment method provided by this invention enables unified installation, deployment, and configuration of multiple nodes on a single node. The script files and program installation packages are specific to the ClickHouse cluster and ZooKeeper distributed automated installation, deployment, and configuration schemes.
[0070] Step S13 involves copying the program to each node using file copying commands. These commands enable remote copying, from local to remote or vice versa. The file copying command can be a secure copy (scp) command or other commands. The scp command is used in Linux for remote file copying. A similar command is cp, but cp only copies files locally and cannot be used across servers, and scp uses encrypted transfers, which may slightly impact speed. When your server's hard drive becomes read-only, scp can help you move files out. Furthermore, scp is very resource-efficient and doesn't significantly increase system load, unlike rsync. While rsync is slightly faster than scp, it can cause very high disk I / O when dealing with many small files, while scp has minimal impact on normal system operation. The scp command format is: scp [parameters][source path][destination path]; its function is to copy files and directories between Linux servers.
[0071] Step S14, establishing a remote connection based on the IP addresses of each node, requires attention to the file copy command in step S13 as a reference when selecting the remote connection method. For example, the `scp` command selected in step S13 above uses the Secure Shell (SSH) protocol, which has the same authentication method, for data transmission to facilitate authentication. The `scp` command is the most convenient and useful command in SSH. For example, transferring files between two servers (i.e., two nodes) can be completely solved with a single `scp` command. You can run `#scp server name: / home / ftp / pub / file1 . / ;` as root on one server, which will directly transfer the file ` / home / ftp / pub / file1;` on the other server to the current directory of the local machine. Similarly, you can use `#scp / tmp / file2 server name: / boot` to send the file ` / tmp / file2` on the local machine to the ` / boot` directory on the other machine, and the entire transfer process is still encrypted using SSH.
[0072] SSH connection principle: The SSH service is a daemon process that listens for client connections in the background. The SSH server process is named sshd, which is responsible for listening for client requests (IP port 22) in real time, including exchanging information such as public keys. The SSH server consists of two parts: openssh (providing the SSH service) and openssl (providing encryption). SSH clients can use tools such as XSHELL, SecureCRT, and Mobaxterm to connect to the SSH working mechanism.
[0073] When the server starts, it generates a 768-bit public key. The local SSH client sends a connection request to the SSH server. The server checks the data and IP address sent by the client, and after confirming their legitimacy, sends the 768-bit public key back to the client. The client then combines its local 256-bit private key and the server's 768-bit public key into a 1024-bit key pair, which it sends back to the server, establishing a connection and transmitting data via key-pair. SSH is a network protocol used for encrypted login between computers. Using the SSH protocol effectively prevents information leakage during remote management. SSH clients are compatible with various platforms, such as Linux, AIX, Solaris, Digital UNIX, Irix, and others. SSH's security is ensured by using asymmetric encryption (RSA) to encrypt all transmitted data.
[0074] The SSH connection establishment process includes the following steps:
[0075] (1) After receiving the user's login request, the remote host sends its public key to the user.
[0076] (2) The user used this public key to encrypt the login password and sent it back.
[0077] (3) The remote host uses its own private key to decrypt the login password. If the password is correct, it will allow the user to log in.
[0078] It's important to note that there are two SSH connection methods: a normal password-protected connection and a passwordless connection. For the passwordless connection, for example, with two servers, one as the server and one as the client, first create a directory named .ssh on the server: `mkdir .ssh`; then assign permissions to the .ssh directory: `chmod 777 .ssh`; next, create a public and private key pair on the client: `ssh-keygen`; finally, copy the client's public key to the server by running the command: `ssh-copy-id root@10.0.0.20`. After entering the password, passwordless SSH access will be achieved.
[0079] Step S15 involves installing and configuring each node using the installer. The installation of ClickHouse and ZooKeeper on each node requires adherence to the installation packages and configuration requirements. This may involve installing the packages, modifying database configurations, or modifying cluster configuration files. The installation and configuration method for each node is not specifically limited; it can be performed simultaneously on all nodes, or the installation and configuration of one node can be completed before proceeding to the next. This invention does not impose any specific limitations.
[0080] In step S16, each node is started in batches using a processing program to achieve installation and deployment. After the installation and deployment of each node are completed, batch startup can be performed. It should be noted that the processing program here is a script file that can achieve batch processing. This invention is not limited to a script file that can achieve batch processing. It can be the xcall.sh script or other scripts. For example, if the xcall.sh script is used, the program can be: . / xcall.sh / cfnet / zookeeper / bin / zkServer.sh start; . / xcall.sh service clickhouse-server start.
[0081] This invention provides a method for installing and deploying a cluster, including obtaining multiple nodes in the cluster system along with their corresponding IPs, script files, and program installation packages; storing the script files and program installation packages on a single current node; copying the program installation packages to each node using a file copy command; establishing remote connections based on the IPs of each node; installing and configuring each node using an installer; and batch starting each node using a processing program to achieve installation and deployment. This method, through an installer and processing program, enables unified installation, deployment, and configuration across multiple nodes using only a single node. It automatically uploads the program installation package via a file copy command, establishes connections between nodes via remote connections, automatically installs and modifies configurations, and then automatically starts the nodes. This avoids individual configuration and operation for each node, greatly simplifying the installation process, saving installation and configuration time, and also solving the problem of missing or disorganized nodes in existing systems.
[0082] Based on the above embodiments, step S15 involves installing and configuring each node using an installer. The installation and deployment method is node-by-node, ensuring seamless transition to the next node after each node's installation and deployment is complete. This includes:
[0083] Get the number of nodes and set the loop count based on the number;
[0084] Install the program's RPM package at the current node, and cycle through the installation of the current node as one installer of the RPM package.
[0085] Determine if the current installer's RPM package loop count exceeds the loop limit;
[0086] If the current installer's RPM package loop count has not exceeded the loop count, then obtain the next current node and return to the step where the installer's RPM package was installed at the current node;
[0087] If the current installer's RPM package loop count exceeds the loop count, return to the current node, install the ClickHouse package of the installer at the current node, modify the database configuration parameters of the current node, and treat the installation of the current node as one loop of installing the ClickHouse package;
[0088] Determine if the current number of times the ClickHouse package is installed exceeds the loop count;
[0089] If the current loop count for installing the ClickHouse package has not exceeded the loop count, then obtain the next current node and return to the steps of installing the ClickHouse package at the current node and modifying the database configuration parameters of the current node;
[0090] If the current loop count for installing the ClickHouse package exceeds the loop count, return to the current node, modify the corresponding cluster configuration file on the current node, and use the configuration of the current node as one configuration loop count;
[0091] Determine if the current configuration loop count exceeds the loop limit;
[0092] If the current configuration loop count has not exceeded the loop count, then obtain the next current node and return to the step of modifying the corresponding cluster configuration file at the current node;
[0093] If the current configuration loop count exceeds the loop count, return to the current node, install ZooKeeper and the corresponding configuration file on the current node, and use the current node's configuration as one ZooKeeper loop count;
[0094] Determine if the current ZooKeeper loop count exceeds the allowed number of loops;
[0095] If the current ZooKeeper loop count has not exceeded the loop count, then obtain the next current node and return to the step of installing ZooKeeper and the corresponding configuration file at the current node;
[0096] If the current ZooKeeper loop count exceeds the loop count, then the installation and configuration of each node are complete.
[0097] Specifically, the installation and deployment of the ClickHouse and ZooKeeper cluster distributed system involves the installation of RPM packages, the ClickHouse package, and the modification of the database configuration parameters of the current node, as well as the installation of ZooKeeper and its corresponding configuration files. The aforementioned configuration and installation parameters can be configured to install all the components on one node first and then start the installation of the next node, or each component can be installed on each node first. This invention does not impose any specific limitations.
[0098] It should be noted that there is no specific restriction on the order of installing the installation and deployment files. You can install the ClickHouse package and modify the database configuration parameters of the current node first, and then install other files.
[0099] Red Hat Package Manager (RPM) is a management program that installs the required software onto a host through a repository management system. RPM packages, developed by Red Hat, are commonly used for installing, upgrading, uninstalling, and querying software packages. Linux uses RPM to implement and manage the installation, viewing, and deletion of software.
[0100] The installation and configuration of this part is completed by installing the ClickHouse package on each node and modifying the database configuration parameters of the current node.
[0101] Modify the corresponding cluster configuration file. Based on the number of nodes and configuration requirements, modify the `metrika.xml` configuration file for each node. Install ZooKeeper and its corresponding configuration file. Install ZooKeeper on each node and modify the `zoo.cfg` and `myid` files accordingly.
[0102] This invention provides a method for installing and deploying the ClickHouse and ZooKeeper cluster distributed system on each node using a cyclical approach. This involves installing the RPM package, the ClickHouse package, modifying the database configuration parameters of the current node, and installing ZooKeeper and its corresponding configuration files. This ensures seamless processing of the installation and configuration of each node, achieving automated installation. `install.sh` handles the installation and configuration, while `xcall.sh` confirms the number of nodes. By simply modifying the number of hosts in the `xcall` script (the number of loops must match the number of input IP nodes), the installation, configuration, and deployment of multiple nodes can be achieved without any limit on the number of nodes, greatly improving work efficiency.
[0103] Based on the above embodiments, after establishing a remote connection based on the IP of each node in step S14, and before installing and configuring each node through the installer in step S15, the method further includes:
[0104] Write the IP address corresponding to each node into the host of each node;
[0105] Perform the basic environment configuration to shut down each node.
[0106] Specifically, after establishing an SSH connection, the node IPs are written into the hosts file of each node, and then the basic environment configuration is shut down so that the installation process is not blocked by the nodes, thus speeding up the installation and configuration efficiency.
[0107] As a preferred example, the basic environment configuration should at least include a firewall and an SELinux configuration package. This is to avoid being blocked from adding defense drivers and to prevent false alarms.
[0108] Security-Enhanced Linux (SELinux) is the National Security Agency's (NSA) implementation of mandatory access control. Under this access control system, processes can only access files necessary for their tasks. SELinux incorporates multi-level security or an optional multi-class policy into a type-enforcement server and employs a role-based control concept.
[0109] SELinux defines many types, and every process, file, device, etc., must identify its type. Processes can only read files of the same type. If there is no relevant type and SELinux does not allow reading, then the file cannot be read. In addition to restricting a process's ability to read files, SELinux also restricts a process's ability to read devices, network connections, communication ports, and inter-process communication, and provides more granular read control.
[0110] The features of the SELinux configuration package are as follows:
[0111] (1) Access control is thorough. Access to all files, directories and ports is based on policy settings that can be configured by the administrator.
[0112] (2) Users are granted only the minimum permissions. Users are divided into roles, and even the root user cannot perform related management tasks without the sysadm_r role. The domains that different roles execute can be modified.
[0113] (3) When SELinux is started, all files and objects have a security context.
[0114] The present invention provides that before each node is installed and deployed, xcall.sh is responsible for confirming the number of nodes. By simply modifying the number of host loops in the xcall program script (the number of loops must be consistent with the number of input IP nodes), the installation, configuration and deployment of multiple nodes can be achieved without any limit on the number of nodes, which greatly improves work efficiency.
[0115] Based on the above embodiments, before installing and configuring each node through the installer in step S15, the method further includes:
[0116] Obtain the server account and password for each node;
[0117] Enter the server account and password into the installation and deployment system of the current node.
[0118] Due to the security nature of SSH connections, it is necessary to obtain the server username and password and enter them into the installation and deployment system of the current node, using SSH encryption. The system automatically connects to each node server via SSH, enters the server username and password, and begins the automated installation, deployment, and configuration process.
[0119] This invention provides a method for obtaining the server account and password corresponding to each node; and inputting the server account and password into the installation and deployment system of the current node. This improves the security of connection transmission and installation and deployment between nodes.
[0120] Based on the above embodiments, step S16, which involves batch starting each node through a processing program, further includes:
[0121] Start the ClickHouse and ZooKeeper programs by batch processing script files.
[0122] It should be noted that the batch processing script here uses the xcall.sh script file. First, it is remotely copied to each node, then permissions are set, and finally, the programs are started in batches. The xcall.sh script file performs the following functions: confirms the number of nodes, establishes SSH connections to each node, and starts the ClickHouse and ZooKeeper programs on each node in batches. The xcall.sh script file is used throughout the entire method embodiment, and its simple implementation process is as follows:
[0123] (1) Select a node server to be installed, create a new folder named cfnet in the root directory, upload the compressed file containing the script and program installation package to this directory, and decompress it;
[0124] (2) Open the unzipped folder and find the xcall.sh script. Users need to modify the number of loops in the script according to the node data to achieve the installation of multiple nodes;
[0125] (3) Based on the known node IP, perform the following operations:
[0126] . / xcall.sh / cfnet / ck_install_shell.19.62 / install.sh IP1 IP2 IP3….IP*;
[0127] (4) Automatically connect to each node server via SSH, enter the server account and password, and start the automated installation, deployment and configuration;
[0128] (5) After installation and configuration, the ClickHouse and ZooKeeper programs on multiple nodes are started using the xcall.sh script. Installation and configuration are complete.
[0129] . / xcall.sh / cfnet / zookeeper / bin / zkServer.sh start
[0130] . / xcall.sh service clickhouse-server start
[0131] This invention provides a method for starting ClickHouse and ZooKeeper programs using batch processing script files. xcall.sh is responsible for confirming the number of nodes. By simply modifying the number of host loops in the xcall script (the number of loops must match the number of input IP nodes), the installation, configuration, and deployment of multiple nodes can be achieved without any limit on the number of nodes, greatly improving work efficiency.
[0132] Based on the above embodiments, the method further includes:
[0133] When nodes are started in batches, a prompt message is output.
[0134] Specifically, during batch startup, a prompt message is output to remind staff that the current installation and configuration work is complete and the next stage of startup can begin, facilitating subsequent startup processes.
[0135] This embodiment provides a prompt message when nodes are started in batches, which helps to remind staff that the current installation and configuration work is completed.
[0136] The foregoing has described in detail the various embodiments corresponding to the installation and deployment of the cluster. Based on this, the present invention also discloses a watch calibration device corresponding to the above-described device. Figure 2 This is a structural diagram of a cluster installation and deployment apparatus provided in an embodiment of the present invention. Figure 2 As shown, the cluster installation and deployment equipment includes:
[0137] Module 11 is used to obtain multiple nodes in the cluster system and their corresponding IPs, script files, and program installation packages;
[0138] Storage module 12 is used to store script files and program installation packages to the current node, where there is only one current node;
[0139] Copy module 13 is used to copy the program installation package to each node via file copy commands;
[0140] Establish module 14 to establish remote connections based on the IP addresses of each node;
[0141] Configuration module 15 is used to install and configure each node via the installer;
[0142] Startup module 16 is used to start each node in batches through a processing program to achieve installation and deployment.
[0143] Since the embodiments of the device part correspond to the embodiments described above, please refer to the embodiments described in the method part for the embodiments of the device part, and will not be repeated here.
[0144] This invention provides a cluster installation and deployment device, comprising: acquiring multiple nodes and their corresponding IPs, script files, and program installation packages in a cluster system; storing the script files and program installation packages on a current node (where the current node is a single node); copying the program installation packages to each node using a file copy command; establishing remote connections based on the IPs of each node; installing and configuring each node using an installer; and batch starting each node using a processing program to achieve installation and deployment. This device, through an installer and processing program, enables unified installation, deployment, and configuration across multiple nodes using only a single node. It automatically uploads the program installation package via a file copy command, establishes connections between nodes via remote connections, automatically installs and modifies configurations, and then automatically starts the nodes. This avoids individual configuration and operation for each node, greatly simplifying the installation process, saving installation and configuration time, and also solving the problem of missing or disorganized nodes in existing systems.
[0145] Figure 3 A structural diagram of another cluster installation and deployment apparatus provided in an embodiment of the present invention is shown below. Figure 3 As shown, the device includes:
[0146] Memory 21 is used to store computer programs;
[0147] Processor 22 is used to implement the steps of installing and deploying a cluster when executing computer programs.
[0148] The installation and deployment device for the seed cluster provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.
[0149] The processor 22 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 22 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 22 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 22 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.
[0150] The memory 21 may include one or more computer-readable storage media, which may be non-transitory. The memory 21 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 21 is used to store at least the following computer program 211, which, after being loaded and executed by the processor 22, is capable of implementing the relevant steps of the seed cluster installation and deployment method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. The operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include, but is not limited to, the data involved in the seed cluster installation and deployment method, etc.
[0151] In some embodiments, the installation and deployment apparatus for the cluster may further include a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27.
[0152] Those skilled in the field can understand, Figure 3 The structure shown does not constitute a limitation on the installation and deployment of the cluster and may include more or fewer components than shown.
[0153] The processor 22 implements the cluster installation and deployment method provided in any of the above embodiments by calling instructions stored in the memory 21.
[0154] This invention provides a cluster installation and deployment device, comprising: acquiring multiple nodes and their corresponding IPs, script files, and program installation packages in a cluster system; storing the script files and program installation packages on a current node (where the current node is a single node); copying the program installation packages to each node using a file copy command; establishing remote connections based on the IPs of each node; installing and configuring each node using an installer; and batch starting each node using a processing program to achieve installation and deployment. This device, through an installer and processing program, enables unified installation, deployment, and configuration across multiple nodes using only a single node. It automatically uploads the program installation package via a file copy command, establishes connections between nodes via remote connections, automatically installs and modifies configurations, and then automatically starts the nodes. This avoids individual configuration and operation for each node, greatly simplifying the installation process, saving installation and configuration time, and also solving the problem of missing or disorganized nodes in existing systems.
[0155] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by processor 22, implements the steps of the cluster installation and deployment method described above.
[0156] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0157] For an introduction to the computer-readable storage medium provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above-mentioned cluster installation and deployment methods.
[0158] The foregoing has provided a detailed description of the installation and deployment method, apparatus, and medium for a seed cluster provided by the present invention. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.
[0159] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
Claims
1. A method for installing and deploying a cluster, characterized in that, include: Obtain multiple nodes in the cluster system, along with their corresponding IPs, script files, and program installation packages; Store the script file and the program installation package in the current node, where the current node is one; The program installation package is copied to each of the nodes using the file copy command; Establish a remote connection based on the IP address of each node; Each node is installed and configured using an installer; the process employs a cyclical approach, where each node is installed and deployed sequentially, seamlessly transitioning to the next node after each node is completed. The nodes are batch-launched using a processing program to achieve installation and deployment; specifically, this includes launching the ClickHouse and ZooKeeper programs using the batch processing script file. Correspondingly, the installation and configuration of each node via the installer includes: Obtain the number of nodes and set the number of loops based on the number; Install the program RPM package of the program installation package at the current node, and cycle through the installation of the program RPM package at the current node as one installation cycle; Determine whether the current number of iterations for installing the program's rpm package exceeds the set number of iterations; If the current number of iterations for installing the program rpm package has not exceeded the number of iterations, then obtain the next current node and return to the step of installing the program rpm package at the current node; If the current number of times the program RPM package is installed exceeds the number of iterations, return to the current node, install the ClickHouse package of the program installation package at the current node, modify the database configuration parameters of the current node, and treat the installation at the current node as one iteration of installing the ClickHouse package; Determine whether the current loop count for installing the ClickHouse package exceeds the loop count; If the current loop count for installing the ClickHouse package does not exceed the loop count, then obtain the next current node and return to the steps of installing the ClickHouse package at the current node and modifying the database configuration parameters of the current node; If the current number of times the ClickHouse package is installed exceeds the loop count, return to the current node, modify the corresponding cluster configuration file at the current node, and use the configuration of the current node as one configuration loop count; Determine whether the current configuration loop count exceeds the specified loop count; If the current configuration loop count has not exceeded the loop count, then obtain the next current node and return to the step of modifying the corresponding cluster configuration file at the current node; If the current configuration loop count exceeds the loop count, return to the current node, install ZooKeeper and the corresponding configuration file at the current node, and use the configuration of the current node as one configuration of the ZooKeeper loop count; Determine whether the current ZooKeeper loop count exceeds the specified loop count; If the current configuration of ZooKeeper does not exceed the number of iterations, then obtain the next current node and return to the step of installing ZooKeeper and the corresponding configuration file at the current node; If the current configuration of ZooKeeper loops exceeds the specified number of loops, then it is determined that the installation and configuration of each node are complete.
2. The cluster installation and deployment method according to claim 1, characterized in that, After establishing a remote connection based on the IP of each node, and before installing and configuring each node using an installer, the method further includes: Write the IP address corresponding to each node into the host of each node; Perform the basic environment configuration to shut down each of the aforementioned nodes.
3. The cluster installation and deployment method according to claim 2, characterized in that, The basic environment configuration includes at least a firewall and an SELinux configuration package.
4. The cluster installation and deployment method according to claim 1, characterized in that, Prior to the installation and configuration of each of the nodes via the installer, the following is also included: Obtain the server account and password corresponding to each node; Enter the server account and password into the installation and deployment system of the current node.
5. The cluster installation and deployment method according to claim 1, characterized in that, Also includes: When the nodes are started in batches, a prompt message is output.
6. A cluster installation and deployment device, characterized in that, include: The acquisition module is used to acquire multiple nodes in the cluster system, their corresponding IPs, script files, and program installation packages. A storage module is used to store the script file and the program installation package to the current node, wherein the current node is one; The copy module is used to copy the program installation package to each of the nodes via file copy instructions; A module is established to create remote connections based on the IP addresses of each node. The configuration module is used to install and configure each of the nodes through the installer; wherein, a cyclical approach is adopted, where each node is installed and deployed one by one, and the next node is processed seamlessly after the previous node is completed; The startup module is used to batch start each of the nodes through a processing program to achieve installation and deployment; specifically, it includes starting the ClickHouse and ZooKeeper programs through the batch processing script file. Correspondingly, the installation and configuration of each node via the installer includes: Obtain the number of nodes and set the number of loops based on the number; Install the program RPM package of the program installation package at the current node, and cycle through the installation of the program RPM package at the current node as one installation cycle; Determine whether the current number of iterations for installing the program's rpm package exceeds the set number of iterations; If the current number of iterations for installing the program rpm package has not exceeded the number of iterations, then obtain the next current node and return to the step of installing the program rpm package at the current node; If the current number of times the program RPM package is installed exceeds the number of iterations, return to the current node, install the ClickHouse package of the program installation package at the current node, modify the database configuration parameters of the current node, and treat the installation at the current node as one iteration of installing the ClickHouse package; Determine whether the current loop count for installing the ClickHouse package exceeds the loop count; If the current loop count for installing the ClickHouse package does not exceed the loop count, then obtain the next current node and return to the steps of installing the ClickHouse package at the current node and modifying the database configuration parameters of the current node; If the current number of times the ClickHouse package is installed exceeds the loop count, return to the current node, modify the corresponding cluster configuration file at the current node, and use the configuration of the current node as one configuration loop count; Determine whether the current configuration loop count exceeds the specified loop count; If the current configuration loop count has not exceeded the loop count, then obtain the next current node and return to the step of modifying the corresponding cluster configuration file at the current node; If the current configuration loop count exceeds the loop count, return to the current node, install ZooKeeper and the corresponding configuration file at the current node, and use the configuration of the current node as one configuration of the ZooKeeper loop count; Determine whether the current ZooKeeper loop count exceeds the specified loop count; If the current configuration of ZooKeeper does not exceed the number of iterations, then obtain the next current node and return to the step of installing ZooKeeper and the corresponding configuration file at the current node; If the current configuration of ZooKeeper loops exceeds the specified number of loops, then it is determined that the installation and configuration of each node are complete.
7. A cluster installation and deployment device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the cluster installation and deployment method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the cluster installation and deployment method as described in any one of claims 1 to 5.