Distributed database deployment method and apparatus, electronic device, and storage medium

Abstract

The application provides a distributed database deployment method and device, electronic equipment and storage medium, relates to the technical field of database, and comprises a management system master control end, a configuration file is created in a file tree, the configuration file is used for configuring an operating system environment of the master node and an operating system environment of the data node, a distributed database installation script of the master node and the data node is created in the file tree, the configuration file is executed based on the management system master control end, communication is performed with the management system controlled end, and the master node and the data node are controlled to configure the operating system environment and execute the distributed database installation script. The method and device provided by the application improve the deployment efficiency of the distributed database.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a distributed database deployment method, apparatus, electronic device, and storage medium. Background Technology

[0002] Greenplum is an open-source big data platform based on a distributed database architecture. It employs an MPP (Massively Parallel Processing) architecture, boasting excellent linear scalability, efficient parallel computing, and storage capabilities. It is suitable for various application scenarios, including efficiently handling petabyte (PB) level data volumes, storage, and real-time analysis.

[0003] Traditional technologies for deploying Greenplum database clusters involve manual deployment across multiple nodes or batch deployment using automated operation and maintenance tools (such as Ansible). Both methods result in long deployment times, poor real-time performance, and low deployment efficiency.

[0004] Therefore, improving the deployment efficiency of distributed databases has become a pressing technical problem for the industry. Summary of the Invention

[0005] This application provides a distributed database deployment method, apparatus, electronic device, and storage medium to address the technical problem of how to improve the deployment efficiency of distributed databases.

[0006] This application provides a distributed database deployment method applied to a computer cluster where a distributed database is to be deployed. The computer cluster includes a master node and data nodes. A management system master control terminal is installed in the master node; a management system slave control terminal is installed in the data nodes. The method includes: The main control terminal of the operation and management system; Create a configuration file in the file tree; the configuration file is used to configure the operating system environment of the master node and the operating system environment of the data nodes; Create a distributed database installation script for the master node and the data nodes in the file tree; Based on the execution of the configuration file by the master control terminal of the management system, communication is established with the controlled terminal of the management system to control the master node and the data node to configure the operating system environment and execute the distributed database installation script.

[0007] In some embodiments, the distributed database is a Greenplum database; the management system is a SaltStack system.

[0008] In some embodiments, the method further includes: Configure passwordless mutual trust between the master control terminal and the controlled terminal of the management system.

[0009] In some embodiments, the method further includes: Configure the node information of the data node in the framework communication protocol configuration file; the node information includes host address information and authentication information; Based on the framework communication protocol configuration file, the managed system controlled terminal is installed in the data node.

[0010] In some embodiments, creating a configuration file in a file tree includes: Create a first configuration file and a second configuration file in the file tree; The first configuration file is used to configure the operating system environment required for the master node to deploy the distributed database; the second configuration file is used to configure the operating system environment required for the data node to deploy the distributed database.

[0011] In some embodiments, the step of executing the configuration file based on the master control terminal of the management system, communicating with the controlled terminal of the management system, controlling the master node and the data node to configure the operating system environment and execute the distributed database installation script includes: The management system's main control terminal receives and executes configuration file instructions. In response to the configuration file execution instruction, the master node is controlled to execute the first configuration file to configure the operating system environment of the master node, and to execute the distributed database installation script of the master node; Based on ZeroMQ, the system communicates with the controlled end of the management system, controls the data node to execute the second configuration file to configure the operating system environment of the data node, and executes the distributed database installation script of the data node.

[0012] In some embodiments, after controlling the master node and the data node to configure the operating system environment and execute the distributed database installation script, the method further includes: Check the log files in the master node and the data nodes; The database status of the distributed database is verified based on the results of the log file inspection.

[0013] This application provides a distributed database deployment device applied to a computer cluster where a distributed database is to be deployed. The computer cluster includes a master node and data nodes. A management system master control terminal is installed in the master node; a management system controlled terminal is installed in the data nodes. The device includes: The runtime module is used to run the main control terminal of the management system. A configuration creation module is used to create configuration files in a file tree; the configuration files are used to configure the operating system environment of the master node and the operating system environment of the data nodes. The script creation module is used to create a distributed database installation script for the master node and the data nodes in the file tree; The deployment module is used to execute the configuration file based on the master control terminal of the management system, communicate with the controlled terminal of the management system, control the master node and the data node to configure the operating system environment and execute the distributed database installation script.

[0014] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the distributed database deployment method described above.

[0015] This application provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the distributed database deployment method described above.

[0016] The distributed database deployment method, apparatus, electronic device, and storage medium provided in this application operate a management system master control terminal; create configuration files in a file tree; the configuration files are used to configure the operating system environments of the master nodes and data nodes; create distributed database installation scripts for the master nodes and data nodes in the file tree; based on the execution of the configuration files by the management system master control terminal, communicate with the management system controlled terminal, and control the master nodes and data nodes to configure the operating system environments and execute the distributed database installation scripts; since the management system master control terminal is installed on the master nodes of the computer cluster and the management system controlled terminal is installed on the data nodes, the configuration of the operating system environments of the master nodes and data nodes and the execution of the distributed database installation scripts are realized through communication between the management system master control terminal and the controlled terminal, realizing the automatic deployment of the distributed database in the computer cluster. During deployment, the master nodes and data nodes can communicate to achieve concurrent execution, improving the deployment efficiency of the distributed database. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the distributed database deployment method provided in this application.

[0020] Figure 2 This is the second flowchart illustrating the distributed database deployment method provided in this application.

[0021] Figure 3 This is a schematic diagram of the structure of the distributed database deployment device provided in this application.

[0022] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps, units, or modules is not necessarily limited to those explicitly listed, but may include other steps, units, or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0025] Enterprise users typically employ ultra-large-scale Greenplum database clusters with dozens, hundreds, or even thousands of nodes to host core data for related business analysis and processing. The initial challenge in this process is how to deploy such large-scale Greenplum clusters.

[0026] One related technology involves manual deployment, starting with configuring the system firewall and time synchronization between the master node and data nodes, and then proceeding through a series of steps including host resolution, kernel parameter tuning, and configuring file connection and process counts. These steps need to be executed one by one, making the entire operation very time-consuming, cumbersome, and labor-intensive. Deployment and switching between multiple nodes is also prone to errors.

[0027] The second related technology involves automatic deployment via Ansible. However, even with a large number of nodes, it can only execute one node at a time, making parallel deployment impossible. Its execution speed is relatively slow and unsuitable for large-scale Greenplum clusters. Furthermore, Ansible's communication efficiency is too low, resulting in relatively poor real-time performance in large-scale production environments.

[0028] In order to address the shortcomings in related technologies, Figure 1 This is one of the flowcharts illustrating the distributed database deployment method provided in this application, such as... Figure 1 As shown, the method includes steps 110, 120, 130 and 140.

[0029] Step 110: Run the main control terminal of the management system.

[0030] Specifically, a computer cluster refers to a computer system that is loosely integrated with a group of computer software or hardware connected together to perform computational tasks in a highly collaborative manner. A computer cluster consists of many nodes. Each node is an independent computer or server, and they are connected together through a network to work together to complete complex computational tasks.

[0031] Distributed databases distribute data across multiple physical computer nodes, which are interconnected via a network to form a logically centralized but physically distributed database system. An example of a distributed database is Greenplum.

[0032] A management system can be used to control and install software on the individual nodes in a computer cluster. For example, a management system could be SaltStack, a centralized server infrastructure management platform with functions such as configuration management, remote execution, and monitoring.

[0033] The distributed database deployment method provided in this application is applied to a computer cluster where a distributed database is to be deployed. The computer cluster includes a master node and data nodes (segment nodes). Typically, there is one master node and multiple data nodes. The master node and data nodes can manage the system. Specifically, the master node manages the system's master control unit, and the data nodes manage the system's minions.

[0034] The distributed database deployment method provided in this application embodiment is executed by a distributed database deployment device. This device can be implemented in software, such as a distributed database deployment program running on the master node management system's main control terminal; it can also be a device that executes the distributed database deployment method, such as a mobile terminal, tablet computer, desktop computer, or server, and can be the master node in a computer cluster.

[0035] A master node can be selected within the computer cluster to manage the system's master control service program. Afterward, the controlled service programs can be managed from the data nodes within the computer cluster.

[0036] Step 120: Create a configuration file in the file tree; the configuration file is used to configure the operating system environment of the master node and the data nodes.

[0037] Specifically, a file tree (also known as a file system tree or directory tree) is a hierarchical structure used to organize and represent files and directories (also known as folders).

[0038] Configuration files can be created in the file tree of the master node. These configuration files primarily include settings for the operating system environment of the master node before deploying the distributed database, as well as settings for the operating system environment of the data nodes before deploying the distributed database.

[0039] The main configuration contents of the configuration file include: configuring the firewall and SELINUX (Security Enhanced Linux, kernel-level security mechanism), time synchronization, HOST (host) resolution, configuring kernel parameters, configuring the number of connections and processes, modifying SSH (Secure Shell Protocol) connections, setting THP (Transparent Huge Pages), disabling IPC (Internet Process Connection) and creating GPADMIN (the default database super administrator in Greenplum database), configuring dynamic link libraries, and configuring environment variables.

[0040] Step 130: Create a distributed database installation script for master nodes and data nodes in the file tree.

[0041] Specifically, a distributed database installation script can be created in the file tree of the master node. This script can be run on both the master and data nodes to perform the installation and configuration steps for the distributed database.

[0042] Step 140: Based on the configuration file executed by the master control terminal of the management system, communicate with the controlled terminal of the management system, control the master node and data node to configure the operating system environment and execute the distributed database installation script.

[0043] Specifically, the master node executes configuration files based on the management system master control terminal, which can control the master node to configure the operating system environment and execute the corresponding distributed database installation scripts; at the same time, the management system master control terminal in the master node can communicate with the management system controlled terminal to control the data nodes to configure the operating system environment and execute the corresponding distributed database installation scripts.

[0044] The distributed database deployment method provided in this application embodiment involves running a management system master control terminal; creating configuration files in a file tree; configuring the operating system environments of the master nodes and data nodes; creating distributed database installation scripts for the master nodes and data nodes in the file tree; and executing the configuration files based on the management system master control terminal to communicate with the management system controlled terminal, controlling the master nodes and data nodes to configure their operating system environments and execute the distributed database installation scripts. Since the management system master control terminal is installed on the master nodes of the computer cluster and the management system controlled terminal is installed on the data nodes, communication between the master control terminal and the controlled terminal enables the configuration of the operating system environments and the execution of the distributed database installation scripts on the master nodes and data nodes, achieving automatic deployment of the distributed database in the computer cluster. During deployment, the master nodes and data nodes can communicate and execute concurrently, improving the deployment efficiency of the distributed database.

[0045] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0046] In some embodiments, the distributed database is a Greenplum database; the management system is a SaltStack system.

[0047] Specifically, the Greenplum database employs a combination of a single master node and multiple segment nodes. The master node is responsible for data management, task coordination, and query optimization, while the segment nodes are responsible for data storage, query execution, and data loading. Through this distributed architecture, Greenplum achieves high performance, high availability, and scalability in large-scale data analysis.

[0048] SaltStack is a centralized server infrastructure management platform implemented in Python, offering features such as configuration management, remote execution, and monitoring. By deploying SaltStack, batch command execution can be performed on tens of millions of servers. It provides centralized configuration management, file distribution, server data collection, operating system and software package management, and more, all tailored to different business needs. SaltStack improves the efficiency of operations and maintenance personnel and standardizes business configuration and operations. The SaltStack system uses a client / server (C / S) model. The server is the SaltStack Master (or SaltMaster node), and the client is the SaltStack Minion (or SaltMinion node). The Salt Master and Salt Minions communicate via ZeroMQ message queues.

[0049] In this embodiment, the Salt Master and Salt Minion of the SaltStack system correspond to the Master node and Segment node of the Greenplum database, respectively. The Salt Master service is deployed on the Master node, and all Segment nodes are configured as hosts within the Salt Master, acting as Salt Minions. The Salt Master then exchanges public keys with the Salt Minions via the ZeroMQ message bus. After this, mutual trust between the Salt Master and Salt Minions is established.

[0050] After mutual trust is established, the Salt Master can send any commands in batches to Salt Minions for execution quickly. Commands sent via the ZeroMQ message bus can be delivered in milliseconds, and multiple Salt Minions can receive normalized commands in parallel. Normalized commands are a series of ordered operations that need to be executed on Greenplum database Segment nodes; these operations are implemented using shell scripts. Using shell scripts to standardize the system operation steps during Greenplum database installation, including installation, configuration, and initialization, facilitates high-volume concurrent execution.

[0051] The distributed database deployment method provided in this application deploys Greenplum databases in large-scale computer clusters. When deploying Greenplum databases on multiple segment nodes based on the SaltStack system, it can achieve efficient parallel execution. Furthermore, since the ZeroMQ message bus is used for communication between nodes, the overall execution efficiency and real-time operation are further improved. The standardized instructions executed by SaltStack reduce the possibility of operational errors during batch operations and improve the deployment success rate.

[0052] In some embodiments, the method further includes: Configure passwordless mutual trust between the master control end and the controlled end of the management system.

[0053] Specifically, in the management system, the Salt SSH protocol (Salt Framework Communication Protocol) is a feature of the SaltStack system used to execute commands via the SSH protocol for remote server management. It allows users to remotely execute commands and manage servers without needing to install the Minion client on the server side.

[0054] After the management system master (Salt Master) in the master node and the management system minion (Salt Minion) in the data node are deployed, passwordless mutual trust is configured between Salt Master and Salt Minion via Salt SSH. After the configuration is completed, verification tests are performed to ensure that all commands sent by the management system master can be executed without obstacles in all management system minions.

[0055] The distributed database deployment method provided in this application configures passwordless mutual trust between the master control terminal and the controlled terminal of the management system, enabling the controlled terminal of the management system to execute the instructions sent by the master control terminal of the management system, thereby improving the deployment efficiency of the distributed database.

[0056] In some embodiments, the method further includes: Configure the node information of the data nodes in the framework communication protocol configuration file; the node information includes host address information and authentication information; Based on the framework communication protocol configuration file, the managed system controlled terminal is installed in the data node.

[0057] Specifically, after installing the management system master terminal on the master node, the node information of the data node can be configured in the framework communication protocol configuration file (Salt SSH protocol configuration file); the node information includes host address information and authentication information.

[0058] By communicating via the Salt SSH protocol, the managed system client can be installed on multiple hosts (as data nodes) in a computer cluster. After the managed system client (Salt Minion) service is installed on all data nodes, the managed system master client on the master node can be started via the Salt SSH protocol.

[0059] The distributed database deployment method provided in this application improves the deployment efficiency of distributed databases by installing a managed system controlled terminal in data nodes through a framework communication protocol configuration file.

[0060] In some embodiments, creating a configuration file in a file tree includes: Create a first configuration file and a second configuration file in the file tree; The first configuration file is used to configure the operating system environment required for the master node to deploy the distributed database; the second configuration file is used to configure the operating system environment required for the data nodes to deploy the distributed database.

[0061] Specifically, a Greenplum State file is created in the file tree of the management system master (Salt Master) on the master node. This file is a configuration file used to define the installation and configuration steps on each data node.

[0062] The configuration files (Greenplum State files) that need to be created include a first configuration file and a second configuration file. The first configuration file, which can be named gp_master.sis, contains the configuration steps for the operating system environment on the master node before installing the Greenplum database. The second configuration file, which can be named gp_segment.sis, contains the configuration steps for the operating system environment on the data nodes before installing the Greenplum database.

[0063] The distributed database deployment method provided in this application creates a first configuration file and a second configuration file in a file tree to configure the operating system environment required for deploying the distributed database on the master node and data nodes, respectively, thereby improving the deployment efficiency of the distributed database.

[0064] In some embodiments, based on the configuration file executed by the master control terminal of the management system, communication is established with the controlled terminal of the management system to control the master node and data nodes to configure the operating system environment and execute the distributed database installation script, including: Based on the management system's main control terminal receiving configuration file execution instructions; In response to the configuration file execution command, control the master node to execute the first configuration file to configure the operating system environment of the master node, and execute the distributed database installation script of the master node; It communicates with the controlled end of the management system based on ZeroMQ, controls the data nodes to execute the second configuration file to configure the operating system environment of the data nodes, and executes the distributed database installation script of the data nodes.

[0065] Specifically, after the configuration file and installation script are created, a configuration file execution command can be sent from the management system master control terminal on the master node. Upon receiving the configuration file execution command, the management system master control terminal responds by controlling the master node to execute the first configuration file to configure the master node's operating system environment and execute the master node's distributed database installation script.

[0066] The management system master terminal also communicates with the management system controlled terminal through ZeroMQ, controlling the data nodes to execute the second configuration file to configure the operating system environment of the data nodes, and executing the distributed database installation script of the data nodes.

[0067] The distributed database deployment method provided in this application improves the deployment efficiency of the distributed database because the master control terminal of the management system communicates with the controlled terminal of the management system through ZeroMQ, enabling data nodes to execute concurrently on a large scale.

[0068] In some embodiments, after controlling the master node and data nodes to configure the operating system environment and execute the distributed database installation script, the method further includes: Check the log files in the master node and data nodes; The database status of the distributed database is verified based on the results of log file inspection.

[0069] Specifically, after installation and configuration are complete, you can use Greenplum database tools and the `gpstate` command to verify the status of the Greenplum database cluster. You can check the log files (Greenplum logs) on the master and data nodes, and based on the results of checking each log file, ensure there are no errors and verify the cluster's connectivity and consistency.

[0070] The distributed database deployment method provided in this application improves the deployment success rate of the distributed database by verifying the database status after the master node and data nodes configure the operating system environment and execute the distributed database installation script.

[0071] Figure 2 This is the second flowchart illustrating the distributed database deployment method provided in this application, as shown below. Figure 2 As shown, this method automates the deployment of the Greenplum database in a computer cluster using the SaltStack system, which employs ZeroMQ message communication bus technology. The method includes: Step 210: Deploy SaltStack Master.

[0072] Install the SaltStack Master service on the selected master node in the computer cluster. During installation, choose either the Debian or RHEL binary package depending on your Linux distribution. After the Salt Master service is installed, start the Master service and configure it to start automatically on boot.

[0073] Step 220: SaltStack Minion Deployment.

[0074] After the SaltStack Master is deployed, the SaltStack Minion service needs to be deployed on each of the N (N>100) Segment nodes.

[0075] Step 230: Configure mutual trust between SaltStack Master and SaltStack Minion.

[0076] After the SaltStack Master and SaltStack Minion are deployed, configure passwordless mutual trust between the SaltStack Master and SaltStack Minion via Salt SSH.

[0077] Step 240: SaltStack State file configuration.

[0078] Create a Greenplum State file in the file tree of the SaltStack Master node to define the installation and configuration steps on each Minion node.

[0079] Step 250: Deploy Greenplum configuration files.

[0080] Create shell scripts in the file tree of the SaltStack Master node to actually install and deploy the Greenplum database on each node. These shell scripts will execute the Greenplum installation and configuration steps, ensuring that any node-specific configuration is handled within the scripts.

[0081] Step 260: SaltStack performs one-click deployment.

[0082] After all the relevant configuration files are configured, execute the State file on the SaltStack Master to trigger the operating system environment configuration before the computer cluster installation, as well as the configuration of the Greenplum database Master and Segments. Based on the definitions in the State file, the SaltStack Master will execute the corresponding installation and configuration steps on each Minion (Segment) node via the Zero MQ message bus.

[0083] Step 270: Verify the Greenplum cluster environment.

[0084] The installation and configuration process across multiple Minion nodes using Salt Master takes roughly the same time as that of a single Minion node. After installation and configuration, use Greenplum's tools and the `gpstate` command to verify the status of the Greenplum database cluster. Check the Greenplum logs on each node to ensure there are no errors and verify the cluster's connectivity and consistency.

[0085] The beneficial effects of the distributed database deployment method provided in this application include: (1) The SaltStack-based architecture enables concurrent processing of Greenplum databases in large-scale computer cluster deployments. In deployments with hundreds or even thousands of nodes, it can efficiently execute deployment operations concurrently, thus decoupling the deployment time of Greenplum databases from the number of nodes.

[0086] (2) The application of ZeroMQ's communication message bus technology reduces the impact of high latency in communication between Master and Segment nodes and command configuration push during deployment on the efficiency of large-scale parallel operations. The execution feedback time is in the millisecond level, which can achieve near real-time deployment process tracking, ensuring real-time tracking effect during deployment and facilitating efficient deployment.

[0087] (3) The standardized instructions (Shell scripts) provide strict rules for configuring multiple operating system environments before Greenplum database installation and initializing multiple databases after installation, avoiding the possibility of errors during batch operations and facilitating large-scale concurrent execution.

[0088] The apparatus provided in the embodiments of this application is described below. The apparatus described below can be referred to in correspondence with the method described above.

[0089] Figure 3 This is a schematic diagram of the structure of the distributed database deployment device provided in this application, as shown below. Figure 3 As shown, this device is applied to a computer cluster where a distributed database is to be deployed. The computer cluster includes master nodes and data nodes; the master node installs the management system's master control terminal; and the data nodes install the management system's controlled terminals. The device includes: Module 310 is used for the main control terminal of the operation management system; The configuration creation module 320 is used to create configuration files in the file tree; the configuration files are used to configure the operating system environment of the master node and the data nodes. Script creation module 330 is used to create distributed database installation scripts for master nodes and data nodes in the file tree; Deployment module 340 is used to execute configuration files based on the master control terminal of the management system, communicate with the controlled terminal of the management system, control the master node and data nodes to configure the operating system environment and execute distributed database installation scripts.

[0090] The distributed database deployment apparatus provided in this application embodiment runs a management system master control terminal; creates configuration files in a file tree; the configuration files are used to configure the operating system environments of the master nodes and data nodes; and creates distributed database installation scripts for the master nodes and data nodes in the file tree. Based on the execution of the configuration files by the management system master control terminal, it communicates with the management system controlled terminal to control the master nodes and data nodes to configure the operating system environments and execute the distributed database installation scripts. Since the management system master control terminal is installed on the master nodes of the computer cluster and the management system controlled terminal is installed on the data nodes, the configuration of the operating system environments and the execution of the distributed database installation scripts for the master nodes and data nodes are realized through communication between the management system master control terminal and the controlled terminal. This realizes the automatic deployment of the distributed database in the computer cluster. During deployment, the master nodes and data nodes can communicate to achieve concurrent execution, which improves the deployment efficiency of the distributed database.

[0091] In some embodiments, the distributed database is a Greenplum database; the management system is a SaltStack system.

[0092] In some embodiments, the configuration creation module is also used for: Configure passwordless mutual trust between the master control end and the controlled end of the management system.

[0093] In some embodiments, the configuration creation module is also used for: Configure the node information of the data nodes in the framework communication protocol configuration file; the node information includes host address information and authentication information; Based on the framework communication protocol configuration file, the managed system controlled terminal is installed in the data node.

[0094] In some embodiments, the configuration creation module is specifically used for: Create a first configuration file and a second configuration file in the file tree; The first configuration file is used to configure the operating system environment required for the master node to deploy the distributed database; the second configuration file is used to configure the operating system environment required for the data nodes to deploy the distributed database.

[0095] In some embodiments, the deployment module is specifically used for: Based on the management system's main control terminal receiving configuration file execution instructions; In response to the configuration file execution command, control the master node to execute the first configuration file to configure the operating system environment of the master node, and execute the distributed database installation script of the master node; It communicates with the controlled end of the management system based on ZeroMQ, controls the data nodes to execute the second configuration file to configure the operating system environment of the data nodes, and executes the distributed database installation script of the data nodes.

[0096] In some embodiments, the deployment module is further configured to: Check the log files in the master node and data nodes; The cluster environment of the distributed database is verified based on the results of the log file inspection.

[0097] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communications bus 440. The processor 410 can call logical commands stored in the memory 430 to execute the methods described in the above embodiments, for example: Run the management system master control terminal; create configuration files in the file tree; the configuration files are used to configure the operating system environment of the master node and the data node; create distributed database installation scripts for the master node and data node in the file tree; based on the execution of the configuration files by the management system master control terminal, communicate with the management system controlled terminal, and control the master node and data node to configure the operating system environment and execute the distributed database installation scripts.

[0098] Furthermore, when the logical commands in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] The processor in the electronic device provided in this application embodiment can call logical instructions in the memory to implement the above method. Its specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effect, which will not be repeated here.

[0100] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0101] The specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, so it will not be repeated here.

[0102] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A distributed database deployment method, characterized in that, A computer cluster for deploying a distributed database, the computer cluster including master nodes and data nodes; The master node installs the management system master control terminal; the data node installs the management system controlled terminal; the method includes: The main control terminal of the operation and management system; Create a configuration file in the file tree; the configuration file is used to configure the operating system environment of the master node and the operating system environment of the data nodes; Create a distributed database installation script for the master node and the data nodes in the file tree; Based on the execution of the configuration file by the master control terminal of the management system, communication is established with the controlled terminal of the management system to control the master node and the data node to configure the operating system environment and execute the distributed database installation script; Creating a configuration file in the file tree includes: Create a first configuration file and a second configuration file in the file tree; The first configuration file is used to configure the operating system environment required for the master node to deploy the distributed database; the second configuration file is used to configure the operating system environment required for the data node to deploy the distributed database. The process of executing the configuration file based on the master control terminal of the management system, communicating with the controlled terminal of the management system, controlling the master node and the data node to configure the operating system environment and execute the distributed database installation script includes: The management system's main control terminal receives and executes configuration file instructions. In response to the configuration file execution instruction, the master node is controlled to execute the first configuration file to configure the operating system environment of the master node, and to execute the distributed database installation script of the master node; Based on ZeroMQ, the system communicates with the controlled end of the management system, controls the data node to execute the second configuration file to configure the operating system environment of the data node, and executes the distributed database installation script of the data node.

2. The distributed database deployment method according to claim 1, characterized in that, The distributed database is Greenplum; the management system is SaltStack.

3. The distributed database deployment method according to claim 1, characterized in that, The method further includes: Configure passwordless mutual trust between the master control terminal and the controlled terminal of the management system.

4. The distributed database deployment method according to claim 1, characterized in that, The method further includes: Configure the node information of the data node in the framework communication protocol configuration file; the node information includes host address information and authentication information; Based on the framework communication protocol configuration file, the managed system controlled terminal is installed in the data node.

5. The distributed database deployment method according to any one of claims 1 to 4, characterized in that, After the master node and the data nodes configure the operating system environment and execute the distributed database installation script, the method further includes: Check the log files in the master node and the data nodes; The database status of the distributed database is verified based on the results of the log file inspection.

6. A distributed database deployment device, characterized in that, A computer cluster for deploying a distributed database, the computer cluster including master nodes and data nodes; The master node is equipped with the management system master control terminal; the data node is equipped with the management system controlled terminal; the device includes: The runtime module is used to run the main control terminal of the management system. A configuration creation module is used to create configuration files in a file tree; the configuration files are used to configure the operating system environment of the master node and the operating system environment of the data nodes. The script creation module is used to create a distributed database installation script for the master node and the data nodes in the file tree; The deployment module is used to execute the configuration file based on the master control terminal of the management system, communicate with the controlled terminal of the management system, control the master node and the data node to configure the operating system environment and execute the distributed database installation script; Creating a configuration file in the file tree includes: Create a first configuration file and a second configuration file in the file tree; The first configuration file is used to configure the operating system environment required for the master node to deploy the distributed database; the second configuration file is used to configure the operating system environment required for the data node to deploy the distributed database. The process of executing the configuration file based on the master control terminal of the management system, communicating with the controlled terminal of the management system, controlling the master node and the data node to configure the operating system environment and execute the distributed database installation script includes: The management system's main control terminal receives and executes configuration file instructions. In response to the configuration file execution instruction, the master node is controlled to execute the first configuration file to configure the operating system environment of the master node, and to execute the distributed database installation script of the master node; Based on ZeroMQ, the system communicates with the controlled end of the management system, controls the data node to execute the second configuration file to configure the operating system environment of the data node, and executes the distributed database installation script of the data node.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the distributed database deployment method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the distributed database deployment method as described in any one of claims 1 to 5.

Images