Database cluster recovery method and related equipment

Abstract

The invention relates to a database cluster recovery method and related equipment. The method comprises the following steps: acquiring metadata information of each node in a to-be-recovered cluster by extracting cluster configuration information and node state information of the to-be-recovered cluster; acquiring backup set data of the to-be-recovered cluster matched with the cluster recovery demand information; determining a node role layout of the target cluster according to the overall operation state of the target cluster; the target cluster is used for recovering the failed to-be-recovered cluster in a cross-environment manner; reconstructing a cluster structure according to the node role layout and the metadata information of each node in the to-be-recovered cluster, distributing backup set data to each node in a target cluster, and executing physical recovery in parallel; and based on each node in the recovered target cluster, adopting a cluster management tool to associate a cluster relationship and executing a cluster registration operation to obtain a recovered new cluster corresponding to the to-be-recovered cluster. By adopting the method, quick equivalent recovery of the database cluster in a cross-environment can be realized, and the cluster recovery efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a database cluster recovery method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] With the wide application of database systems in informationization, finance, energy and other fields, higher requirements are put forward for the high availability and reliability of database systems.

[0003] In the related art, traditional database high availability solutions, such as master-slave replication, failover and other technologies, are relatively mature in fault switching, but there are still deficiencies in data consistency recovery, node reconstruction and system self-healing after failure. Most cluster systems still rely on manual intervention to complete environment recovery and node rejoining after node failure, resulting in complex and time-consuming recovery process, and prone to human configuration errors. SUMMARY

[0004] Therefore, it is necessary to provide a database cluster recovery method, device, computer equipment, computer readable storage medium and computer program product to solve the above technical problems.

[0005] In a first aspect, the present application provides a database cluster recovery method, which comprises:

[0006] By extracting the cluster configuration information and node state information of the to-be-recovered cluster, the metadata information of each node in the to-be-recovered cluster is obtained; the metadata information at least includes the node role and connection parameters of the corresponding node, and the node running state; the node role includes master node and slave node;

[0007] Backup set data of the to-be-recovered cluster matched with the cluster recovery requirement information is obtained; the backup set data is the database business data written by the master node in the to-be-recovered cluster;

[0008] According to the overall running state of the target cluster, the node role layout of the target cluster is determined; the target cluster is used to recover the to-be-recovered cluster which has failed across environments;

[0009] According to the node role layout and the metadata information of each node in the to-be-recovered cluster, the cluster structure is reconstructed, the backup set data is distributed to each node in the target cluster, and the physical recovery is performed in parallel;

[0010] Based on each node in the recovered target cluster, the cluster management tool is used to associate the cluster relationship and perform the cluster registration operation, and the new cluster corresponding to the to-be-recovered cluster after recovery is obtained.

[0011] In one of the embodiments, the obtaining the backup set data of the cluster to be recovered matching the cluster recovery requirement information comprises:

[0012] According to the cluster recovery requirement information, a specified recovery time point is obtained;

[0013] In the backup server, a backup set matching the recovery time point is queried, and the backup set data of the cluster to be recovered is obtained.

[0014] In one of the embodiments, the determining the node role layout of the target cluster according to the overall running state of the target cluster comprises:

[0015] According to the overall running state of the target cluster, it is determined that each node and its instance in the target cluster are running normally;

[0016] According to the current master-slave node information of the target cluster, the node role layout is obtained.

[0017] In one of the embodiments, the determining the node role layout of the target cluster according to the overall running state of the target cluster comprises:

[0018] According to the overall running state of the target cluster, it is determined that the target cluster is in an unavailable state;

[0019] According to the cluster metadata information of the target cluster extracted in advance, the node role layout is obtained.

[0020] In one of the embodiments, the distributing the backup set data to each node in the target cluster and performing physical recovery in parallel comprises:

[0021] The backup set data is distributed to each node in the target cluster;

[0022] Through the recovery operation performed by each node in the target cluster at the same time, the original data of each node in the target cluster is covered by the backup set data.

[0023] In one of the embodiments, the method further comprises:

[0024] Starting the new cluster after recovery;

[0025] Performing a node role judgment operation according to the running health state of the new cluster after recovery to allocate the master node and the slave node of the new cluster after recovery.

[0026] In a second aspect, the application further provides a database cluster recovery device, the device comprises:

[0027] The metadata extraction module is configured to obtain metadata information of each node in the cluster to be recovered by extracting cluster configuration information and node state information of the cluster to be recovered; the metadata information at least includes a node role and connection parameters of a corresponding node, and a node running state; the node role includes a master node and a slave node;

[0028] The backup set acquisition module is configured to acquire backup set data of the cluster to be recovered matched with the cluster recovery requirement information; the backup set data is database business data written by a master node in the cluster to be recovered;

[0029] The node role layout determination module is configured to determine a node role layout of the target cluster according to an overall running state of the target cluster; the target cluster is used to recover the cluster to be recovered which has occurred a failure in a cross-environment;

[0030] The parallel recovery module is configured to reconstruct a cluster structure according to the node role layout and the metadata information of each node in the cluster to be recovered, distribute the backup set data to each node in the target cluster, and perform a physical recovery in parallel.

[0031] The cluster registration module is configured to associate a cluster relationship and perform a cluster registration operation by using a cluster management tool based on each node in the target cluster after recovery, so as to obtain a new cluster after recovery corresponding to the cluster to be recovered.

[0032] In a third aspect, the present application further provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0033] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0034] In a fifth aspect, the present application further provides a computer program product, including a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0035] The database cluster recovery method, device, computer equipment, computer readable storage medium and computer program product, by extracting the cluster configuration information and node state information of the to-be-recovered cluster, obtaining the metadata information of each node in the to-be-recovered cluster, the metadata information at least including the node role and connection parameter of the corresponding node, and the node running state, the node role including the master node and the slave node, obtaining the backup set data of the to-be-recovered cluster matched with the cluster recovery requirement information, the backup set data being the database business data written by the master node in the to-be-recovered cluster, then determining the node role layout of the target cluster according to the overall running state of the target cluster, the target cluster being used to recover the to-be-recovered cluster which fails across environments, reconstructing the cluster structure according to the node role layout and the metadata information of each node in the to-be-recovered cluster, distributing the backup set data to each node in the target cluster, and executing the physical recovery in parallel, and then based on each node in the recovered target cluster, using the cluster management tool to associate the cluster relationship and execute the cluster registration operation, obtaining the recovered new cluster corresponding to the to-be-recovered cluster, the database cluster recovery scheme is optimized, the cluster structure is accurately restored based on the metadata, the recovery time is shortened, and the master-slave relationship and self-healing capability are guaranteed by using the cluster management tool, which not only ensures the integrity and consistency of the business data, but also improves the fault recovery efficiency and cluster availability, and can realize the fast equivalent recovery of the database cluster across environments, and meets the high continuity business demand. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other related drawings can be obtained by those skilled in the art without creative labor.

[0037] Figure 1 It is a schematic diagram of a traditional technical framework in an embodiment;

[0038] Figure 2 It is a flowchart of a database cluster recovery method in an embodiment;

[0039] Figure 3 It is a schematic diagram of the recovery technical framework of the present application in an embodiment;

[0040] Figure 4 It is a schematic diagram of a database cluster recovery processing flow in an embodiment;

[0041] Figure 5 It is a flowchart of a database cluster recovery method in another embodiment;

[0042] Figure 6A structural block diagram of a database cluster recovery device in an embodiment;

[0043] Figure 7 An internal structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0045] As shown in Figure 1 for the traditional database high-availability solution:

[0046] 1. The scheme based on the cloned master node, which realizes the recovery of the standby node by cloning the master node, first empties the instance directory of the standby node, and then copies the instance data directory and all custom table space files of the master node to the file system of the standby node. In actual application, if the data volume of the master node is large, the file copying process will consume a lot of time and reduce the recovery efficiency; this scheme is not suitable for production environments with large data size or high recovery timeliness requirements.

[0047] 2. The backup set copying scheme based on full backup, which relies on the physical backup set generated by the source cluster master node to complete the recovery operation in the target cluster. When the target cluster is not the source cluster itself, i.e. cross-cluster recovery, it is easy to cause cluster information pollution problem; the database instance after recovery will retain the cluster configuration information such as IP address of each node in the original cluster, which leads to the failure of the recovery environment to run normally, affecting the stability and availability of the source cluster. And because the design strongly binds the backup data with the source cluster environment, the backup set can only be applied to the current cluster during recovery, and cannot be flexibly migrated and deployed in different servers or heterogeneous cluster environments, limiting its versatility and practicality.

[0048] 3. The master-standby node scheme based on manual determination. The traditional master-standby node recovery process usually relies on manual judgment and manual designation of master-standby node roles, which is low in efficiency and easy to make mistakes.

[0049] Therefore, how to realize an automatic and low-intervention cluster recovery mechanism while ensuring the continuity of database business is an important research direction for improving the reliability of current database systems.

[0050] To solve the above problems, the application provides a master-slave cluster fault recovery and data repair method suitable for PostgreSQL (a relational database management system) and its derivative databases, which has an automatic cluster recovery and self-healing mechanism for production environment, i.e., a database master-slave cluster physical recovery and self-healing based on Repmgr, and can solve the problems of low automation, dependence on manual operation, and interference with the source database in the process of recovering the master-slave cluster in a heterogeneous environment by traditional technology. The overall scheme of the application considers both efficiency and safety, and can be applied to key database cluster scenarios that need to be quickly restored to a heterogeneous environment.

[0051] To better understand the embodiments of the present disclosure by those skilled in the art, the technical terms or partial nouns that the embodiments of the present application may involve are explained as follows:

[0052] PostgreSQL database: It is an open source, powerful relational database management system, which is known for its high compliance with SQL standards, support for transaction consistency and rich data types. Due to its good scalability, stability and active community ecology, it has been widely used in financial, energy, information and other key fields.

[0053] Repmgr: It is a high availability management tool supported by PostgreSQL, which is used to manage the master-slave replication environment (streaming replication) of PostgreSQL, can provide master-slave node registration, fault detection, automatic switching (failover), manual / automatic rejoining of cluster and other functions, and is an important component for building a high-availability PostgreSQL cluster. It includes a daemon (repmgrd) and a command line tool, which can be used together to realize role recognition and switching management between master and slave databases.

[0054] Agent: A lightweight detection and execution component deployed on each node in the database cluster, which is used to collect the instance running state, master-slave role information, process configuration (such as repmgr configuration path, PostgreSQL instance state) of the local node. It is a key execution layer for realizing automatic recovery, self-healing and unified management of the database cluster, can receive backup server tasks and feedback results, realize real-time sensing of node state and automatic distribution of management operations.

[0055] Master-slave cluster: A database deployment architecture consisting of one primary node (Primary) and one or more standby nodes (Standby), designed to achieve high availability and data redundancy. In this structure, the primary node handles read and write requests, while the standby nodes maintain data consistency with the primary node through replication mechanisms. In the event of a primary node failure, a failover mechanism can be used to promote a standby node to a new primary node, ensuring business continuity. PostgreSQL can be combined with repmgr to build a master-slave cluster.

[0056] Failover: In a master-slave cluster architecture, when the primary node fails or becomes unavailable, the system automatically or manually switches business to a standby node, which takes over the primary node's responsibilities. During the failover process, it involves role conversion, client redirection, data consistency verification, and other aspects, which is an important mechanism to ensure the high availability of the database system. The repmgr tool can support automatic fault detection and master-slave switching, minimizing business interruption.

[0057] Master-slave replication: A database cluster architecture used to achieve high availability and read-write separation of data. In this architecture, one database node is designated as the "master node" (Master), responsible for handling all write operations (such as insert, update, delete); one or more other nodes are designated as "slave nodes" (Slave or Standby), responsible for synchronizing data replicas from the master node, usually only handling read operations. In PostgreSQL, master-slave replication can be achieved through physical replication (such as log streaming), and slave nodes maintain data consistency with the master node by continuously receiving and applying transaction logs from the master node. With the help of tools such as repmgr, master-slave role management, automatic failover, and role promotion can be achieved in the cluster, thereby improving the stability and disaster recovery capabilities of the cluster.

[0058] In an exemplary embodiment, as shown in Figure 2 a database cluster recovery method is provided, and the embodiment is applied to a system including a terminal and a server, such as a database cluster management system, and is realized through the interaction of the terminal and the server. In this embodiment, the method includes the following steps 201 to 205. Among them:

[0059] Step 201: Obtain the metadata information of each node in the to-be-restored cluster by extracting the cluster configuration information and node state information of the to-be-restored cluster.

[0060] As an example, the metadata information at least includes the node role and connection parameters of the corresponding node, and the node running state; the node role includes the primary node and the slave node.

[0061] In actual application, for each node in the cluster to be recovered due to database failure, an agent (Agent) deployed on each node can detect and collect metadata of a local database instance of each node, each instance corresponding to a node in the cluster, and can store the obtained relevant metadata information on the host where the node is located.

[0062] For example, the metadata information of each node can be used as an important basis for cluster reconstruction in the subsequent cluster recovery process, which can include but is not limited to cluster configuration file, cluster type, cluster management tool installation path, master node IP, slave node IP, and further through extracting the configuration information (such as node role, connection parameter, user permission, etc.) and node running state of the source database cluster (i.e. the cluster to be recovered), the complete modeling of the cluster structure and the restoration of the information before recovery can be achieved.

[0063] In an optional embodiment, the local database instance metadata of the target cluster (which is used to recover the cluster to be recovered due to failure across environments) can also be extracted for subsequent use in identifying the master and standby nodes of the target cluster when the target cluster is in an abnormal running state.

[0064] Step 202: Obtain backup set data of the cluster to be recovered matched with the cluster recovery requirement information.

[0065] The backup set data is the database business data written by the master node in the cluster to be recovered.

[0066] In a specific implementation, the corresponding backup set (i.e. backup set data of the cluster to be recovered matched with the cluster recovery requirement information) can be filtered and obtained from the backup storage (storaged) according to the cluster recovery requirement information; since the master node is responsible for write operation, the data is the most complete and reliable, and the backup of the master node is more referential, and thus each backup set data comes from the master node of the cluster.

[0067] Optionally, the backup set data can include data directory, table space and other key data of the database instance; which can be used to write data to the corresponding position of the target cluster in the subsequent cluster recovery process.

[0068] Step 203: Determine the node role layout of the target cluster according to the overall running state of the target cluster; the target cluster is used to recover the cluster to be recovered due to failure across environments.

[0069] In an example, before performing the cluster recovery operation, the database cluster management system can determine the node role layout of the target cluster, such as the master node and the slave node, by judging the running state of the target cluster; thereby based on the automatic judgment of the role of each node in the target cluster during the overall recovery process, the system can automatically select the appropriate master-slave node layout (i.e. node role layout) according to the overall running state, and can further reset the original registration information of the cluster, effectively avoiding the conflict with the source end.

[0070] Step 204, reconstruct the cluster structure according to the node role layout and the metadata information of each node in the cluster to be recovered, distribute the backup set data to each node in the target cluster, and perform physical recovery in parallel.

[0071] Specifically, during the cluster recovery process, the backup server (backupd) can obtain the backup set from the backup storage server (storaged) and distribute it to each node of the target cluster, and then each node can perform physical recovery according to the received data. Thus, at the level of recovery execution, the present method breaks the traditional serial process of first restoring the master node and then cloning the standby node, and by using the operation of parallel execution of physical recovery by each node, the recovery process time can be effectively shortened.

[0072] Step 205, based on each node in the target cluster after recovery, use a cluster management tool to associate the cluster relationship and perform a cluster registration operation to obtain the new cluster after recovery corresponding to the cluster to be recovered.

[0073] In an example, after the recovery is completed, the cluster relationship can be rebuilt by the repmgr tool (i.e. cluster management tool) to ensure the stability of the master-slave structure of the new cluster, and to avoid the pollution of the residual information of the old cluster (i.e. target cluster) to the current cluster (i.e. new cluster after recovery). Compared with the traditional method, the technical solution of the present embodiment, as shown in Figure 3 can realize the simultaneous recovery of each node in the cluster, can avoid the situation of using the slave node to clone the master node one by one after the master node is recovered, and effectively improves the recovery efficiency.

[0074] In the database cluster recovery method, the cluster configuration information and the node state information of the to-be-recovered cluster are extracted to obtain the metadata information of each node in the to-be-recovered cluster, the backup set data of the to-be-recovered cluster matched with the cluster recovery requirement information is obtained, the node role layout of the target cluster is determined according to the overall running state of the target cluster, the cluster structure is reconstructed according to the node role layout and the metadata information of each node in the to-be-recovered cluster, the backup set data is distributed to each node in the target cluster, the physical recovery is performed in parallel, then the cluster relationship is associated and the cluster registration operation is performed based on each node in the recovered target cluster by using the cluster management tool, and the recovered new cluster corresponding to the to-be-recovered cluster is obtained, so that the database cluster recovery scheme is optimized, the cluster structure is accurately restored based on the metadata, the recovery time is shortened, the master-slave relationship is guaranteed by using the cluster management tool, and the self-recovery capability is improved, so that the business data is ensured to be complete and consistent, the fault recovery efficiency and the cluster availability are improved, the fast equivalent recovery of the database cluster under cross-environment is realized, and the high-continuity business requirement is met.

[0075] In an exemplary embodiment, the backup set data of the to-be-recovered cluster matched with the cluster recovery requirement information can include the following steps:

[0076] According to the cluster recovery requirement information, a specified recovery time point is obtained; a backup set matched with the recovery time point is queried in the backup server, and the backup set data of the to-be-recovered cluster is obtained.

[0077] In actual application, as shown in Figure 4 The database cluster management system can include a backup server (backupd) for task scheduling and management of all client resources, storage server, backup and recovery job data, and a backup storage server (storaged) for storing cluster backup data. For example, the backup server can send instructions to the backup storage server to obtain the specified backup set according to the user's backup set acquisition request. The backup storage server can find the corresponding backup data according to the storage path and related metadata of the backup set, and can transmit it to the backup server. Then the backup server can restore the backup set to the target location of the specified cluster to complete the data recovery operation.

[0078] For example, as shown in Figure 4 The backup server can query the backup set according to the recovery time point in the request and return the backup set query result, i.e., the backup set data of the to-be-recovered cluster, in response to the user's backup set acquisition request (i.e., the cluster recovery requirement information).

[0079] In this embodiment, by obtaining the specified recovery time point according to the cluster recovery demand information, and then querying the backup set matching the recovery time point in the backup server to obtain the backup set data of the cluster to be recovered, the required backup set can be accurately located, the pre-recovery time consumption is shortened, the consistency of the recovered data and the target time point is guaranteed, and the cluster recovery accuracy and efficiency are effectively improved.

[0080] In one example embodiment, the determining the node role layout of the target cluster according to the overall running state of the target cluster can include the following steps:

[0081] According to the overall running state of the target cluster, it is determined that each node and its instance in the target cluster is running normally; and according to the current master-slave node information of the target cluster, the node role layout is obtained.

[0082] In an example, if it is determined that the running state of the target cluster is normal, it can be indicated that the cluster nodes and instances are running normally, and then the current master-slave node information can be directly obtained to perform the recovery operation, and the master-slave roles of each node remain unchanged after the recovery is completed.

[0083] In this embodiment, by determining that each node and its instance in the target cluster is running normally according to the overall running state of the target cluster, and then obtaining the node role layout according to the current master-slave node information of the target cluster, the accurate matching of the cluster role layout can be realized, the recovery abnormality caused by state misjudgment is avoided, the manual intervention cost is reduced, and a reliable data foundation is provided for the cluster self-healing mechanism.

[0084] In one example embodiment, the determining the node role layout of the target cluster according to the overall running state of the target cluster can include the following steps:

[0085] According to the overall running state of the target cluster, it is determined that the target cluster is in an unusable state; and according to the cluster metadata information of the target cluster extracted in advance, the node role layout is obtained.

[0086] In an example, if it is determined that the running state of the target cluster is abnormal, it can be indicated that the cluster is in an unusable state, such as the case that some instances in the cluster are damaged, at which time reliable master-slave node information cannot be dynamically obtained; and then the local database instance metadata of the target cluster collected in advance can be used to identify the master-slave nodes, so that the fault tolerance capability can be enhanced and the self-healing recovery of the cluster can be supported. For example, the master-slave roles (i.e., the node role layout) detected by the most initial collection can be recovered.

[0087] In the embodiment, the target cluster is determined to be in an unavailable state according to the overall running state of the target cluster, and then the node role layout is obtained according to the cluster metadata information of the target cluster extracted in advance, so that the node role can be quickly located in the cluster abnormal situation, the duration of the unavailable state is shortened, a data basis is provided for further automatic recovery, and the fault response efficiency is effectively improved.

[0088] In an exemplary embodiment, the distributing the backup set data to each node in the target cluster and performing the physical recovery in parallel can include the following steps:

[0089] The backup set data is distributed to each node in the target cluster, and the recovery operation is performed by each node in the target cluster at the same time to overwrite the original data of each node in the target cluster with the backup set data.

[0090] In actual application, as shown in Figure 4 The database cluster management system can also include an agent deployed on each node. The agent can parse the recovery job parameters by receiving the recovery request sent by the backup server, obtain the specific information of the source cluster (i.e., the cluster to be recovered) and the target cluster, such as the node role layout of the target cluster and the metadata information of each node in the cluster to be recovered. Then, the database instances in the target cluster can be stopped, such as stopping each database node in the target cluster. Further, based on the backup set (i.e., backup set data) obtained from the backup set storage pool, the agent can control each node to perform the recovery operation at the same time, and use the data in the backup set to overwrite the original data of each node in the target cluster.

[0091] In an example, as shown in Figure 4 The master node can be started and registered to the cluster by the agent, and the slave node can be controlled to perform the recovery registration operation to register to the cluster where the master node is located. The registration result can be returned to the agent by the database cluster to determine whether the registration is successful. If the registration fails, manual operation commands can be provided in the log.

[0092] In the embodiment, the backup set data is distributed to each node in the target cluster, and then the recovery operation is performed by each node in the target cluster at the same time to overwrite the original data of each node in the target cluster with the backup set data, so that the cluster data consistency recovery can be achieved.

[0093] In an exemplary embodiment, the following steps can also be included:

[0094] Starting the new cluster after recovery; performing a node role judgment operation according to the running health state of the new cluster after recovery to allocate the master node and the slave node of the new cluster after recovery.

[0095] In a specific implementation, in order to improve the intelligent and automatic level of the recovery process, a mechanism for automatically identifying and assigning primary and standby nodes based on target cluster health state information can be introduced. This mechanism can automatically determine the instance that is most suitable for serving as a primary node by detecting the running state, synchronization delay, load condition and other indicators of each node, and complete the primary and standby relationship configuration accordingly, thereby improving the recovery efficiency and reducing the risk of human intervention. For example, after the cluster is started, the database cluster management system can automatically determine and assign the primary and standby node roles according to the running health state of the target cluster, so as to avoid human intervention and realize the dynamic construction of the primary and standby relationship and the cluster self-healing mechanism.

[0096] Compared with the traditional method, the technical solution of the embodiment avoids the traditional serial process of “restoring the primary node first and then cloning the standby node”, can greatly shorten the overall recovery time when facing large clusters, effectively reduces the risk of human operation errors and improves the standardization and reliability of the recovery process by automatically parsing and restoring the configuration and state information of the source cluster, can avoid interference with the source cluster and protect the stable operation and data security of the source system without relying on the source primary node or performing cross-primary and standby cloning operations, can make the recovered cluster have a complete primary and standby structure and automatic fault recovery capability by automatically rebuilding cluster information, dynamically assigning primary and standby roles, and executing the repmgr command to realize cluster registration and primary and standby synchronization, and can realize fast deployment and recovery in a different place or different machine environment, so that the technical solution of the embodiment is more suitable for various practical business scenarios such as disaster recovery, migration and testing.

[0097] In order for those skilled in the art to better understand the above steps, the embodiments of the present application are exemplarily described below by an example, but it should be understood that the embodiments of the present application are not limited thereto.

[0098] The following case is taken as an example for illustration: the cluster to be recovered is a repmgr primary and standby cluster (primary node A, standby node B, standby node C) deployed, and the target cluster is 3 new servers (target nodes D, E, F) which have pre-installed repmgr tools and each node is deployed with an Agent component.

[0099] 1. Cluster metadata acquisition process: automatically start the metadata detection task through the Agent, read the cluster configuration file from the surviving original standby node C, extract the cluster type as Reprgr, and obtain the Reprgr installation path, and identify the IP and user permission configuration of the original cluster primary node A, standby node B and standby node C; store the above metadata in the local directory of the target nodes D, E and F respectively, and obtain a metadata set containing complete cluster structure information, which provides a basis for subsequent recovery.

[0100] 2. Backup set data acquisition process: The cluster to be restored adopts a daily full and real-time incremental backup strategy. The full backup set is stored in an off-site backup storage end, and the backup set is from the original cluster master node. The operation and maintenance personnel can initiate a task through the recovery management platform. The database cluster management system can automatically filter the matching backup set in the backup storage end and check the backup set integrity according to the requirement of "restoring to 10 minutes before the power failure". The backup storage end can transmit the filtered backup set to the backup service end.

[0101] 3. Judgment of target cluster state and execution of recovery process: The backup service end can detect the running state of the target nodes D, E and F. If the database instances of the nodes are not started and no master-slave relationship is configured, it is determined as an abnormal state, and the recovery needs to be executed based on the metadata extracted in the cluster metadata acquisition process. For example, the backup service end can simultaneously distribute the backup set to the target nodes D, E and F, so that the three nodes receive data in parallel. Then, the local instance directory of each target node can be emptied, and the default configuration file is deleted to avoid interference of old data. Then, the target nodes D, E and F can execute physical recovery based on the received backup set, write the data directory and table space file in the backup set into the local instance directory, and simultaneously restore the configuration according to the metadata (such as modifying the target node IP) to obtain three target nodes that complete physical recovery. At this time, the instance is started but the cluster is not formed.

[0102] 4. Rebuilding of cluster relationship and implementation of self-healing process: Each target node can initiate cluster registration through the repmgr tool. The system reads the original cluster master-slave role information in the metadata, and can automatically set the target node D closest to the original master node A as the candidate master node according to the current health state of the target node (such as CPU load, memory usage, etc.), and execute the repmgr command to register E and F as the standby nodes of D to complete the rebuilding of the master-slave relationship. The system automatically deletes the original cluster node IP and cluster ID remaining in the backup set, and generates a new cluster identifier to avoid conflict with the original cluster information.

[0103] Self-healing verification: After the recovery is completed, the system can simulate the failure of the target node D. The Repmgr daemon detects the failure and can automatically calculate the synchronization delay and hardware load (such as lower load of E) of the nodes E and F. The node E can be promoted as the new master node, and the node F can be automatically re-registered as the standby node of E to verify that the cluster has self-healing capability.

[0104] Business switching: The operation and maintenance personnel can switch the database connection address of the system from the original cluster IP to the new master node E after confirming that the cluster state is normal. The business is restored to normal, and the total time from failure to business recovery is short, which meets the high availability requirement.

[0105] In an exemplary embodiment, as Figure 5As shown, a flow diagram of another database cluster recovery method is provided. In this embodiment, the method comprises the following steps:

[0106] In step 501, metadata information of each node in the cluster to be recovered is obtained by extracting cluster configuration information and node state information of the cluster to be recovered. In step 502, a specified recovery time point is obtained according to cluster recovery requirement information, a backup set matching the recovery time point is queried in the backup server, and backup set data of the cluster to be recovered is obtained. In step 503, the node role layout of the target cluster is determined according to the overall running state of the target cluster. In step 504, the cluster structure is reconstructed according to the node role layout and the metadata information of each node in the cluster to be recovered. In step 505, the backup set data is distributed to each node in the target cluster, and the original data of each node in the target cluster is overwritten by the backup set data through the simultaneous execution of the recovery operation by each node in the target cluster. In step 506, based on each node in the target cluster after recovery, the cluster management tool is used to associate the cluster relationship and perform the cluster registration operation, and the new cluster after recovery corresponding to the cluster to be recovered is obtained.

[0107] It should be noted that the specific limitations of the above steps can be referred to the specific limitations of the above-mentioned database cluster recovery method, which will not be repeated here.

[0108] It should be understood that although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise stated herein, the execution of these steps has no strict order limitation, and these steps can be executed in other order. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0109] Based on the same inventive concept, the present embodiment also provides a database cluster recovery device for implementing the above-mentioned database cluster recovery method. The implementation scheme of the device for solving the problem is similar to the implementation scheme described in the above method, so the specific limitations in one or more database cluster recovery device embodiments provided below can be referred to the limitations of the database cluster recovery method in the above text, which will not be repeated here.

[0110] In one exemplary embodiment, as Figure 6 shown, a database cluster recovery device is provided, comprising:

[0111] The metadata extraction module 601 is configured to obtain metadata information of each node in the cluster to be recovered by extracting cluster configuration information and node state information of the cluster to be recovered; the metadata information at least includes a node role and connection parameters of a corresponding node, and a node running state; the node role includes a master node and a slave node;

[0112] The backup set acquisition module 602 is configured to acquire backup set data of the cluster to be recovered matched with the cluster recovery requirement information; the backup set data is database business data written by a master node in the cluster to be recovered;

[0113] The node role layout determination module 603 is configured to determine a node role layout of the target cluster according to an overall running state of the target cluster; the target cluster is used to recover the cluster to be recovered which has occurred a failure in a cross-environment;

[0114] The parallel recovery module 604 is configured to reconstruct a cluster structure according to the node role layout and the metadata information of each node in the cluster to be recovered, distribute the backup set data to each node in the target cluster, and perform a physical recovery in parallel.

[0115] The cluster registration module 605 is configured to associate a cluster relationship and perform a cluster registration operation based on each node in the target cluster after recovery by using a cluster management tool, and obtain a new cluster after recovery corresponding to the cluster to be recovered.

[0116] In one of the embodiments, the backup set acquisition module 602 is specifically configured to acquire a specified recovery time point according to the cluster recovery requirement information; and query a backup set matched with the recovery time point in a backup service end, and acquire the backup set data of the cluster to be recovered.

[0117] In one of the embodiments, the node role layout determination module 603 is specifically configured to determine that each node and an instance thereof in the target cluster are running normally according to the overall running state of the target cluster; and acquire the node role layout according to current master-slave node information of the target cluster.

[0118] In one of the embodiments, the node role layout determination module 603 is specifically configured to determine that the target cluster is in an unusable state according to the overall running state of the target cluster; and acquire the node role layout according to the cluster metadata information of the target cluster extracted in advance.

[0119] In one of the embodiments, the parallel recovery module 604 is specifically configured to distribute the backup set data to each node in the target cluster; and simultaneously perform the recovery operation by each node in the target cluster to overwrite the original data of each node in the target cluster with the backup set data.

[0120] In one of the embodiments, the apparatus further comprises:

[0121] a node role allocation module configured to start the new cluster after recovery; and perform a node role judgment operation according to the running health status of the new cluster after recovery to allocate the master node and the slave node of the new cluster after recovery.

[0122] The modules in the database cluster recovery apparatus can be realized by software, hardware, or a combination thereof. The modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the modules.

[0123] In one of the exemplary embodiments, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in Figure 7 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store database cluster recovery data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a database cluster recovery method.

[0124] Those skilled in the art can understand that Figure 7 the structure shown in the above

[0125] In one exemplary embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0126] By extracting cluster configuration information and node state information of the to-be-restored cluster, metadata information of each node in the to-be-restored cluster is obtained; the metadata information at least includes node role and connection parameter of the corresponding node, and node running state; the node role includes master node and slave node;

[0127] Backup set data of the to-be-restored cluster matched with the cluster recovery requirement information is obtained; the backup set data is database business data written by the master node in the to-be-restored cluster;

[0128] According to the overall running state of the target cluster, the node role layout of the target cluster is determined; the target cluster is used to restore the to-be-restored cluster which has failed across environments;

[0129] According to the node role layout and the metadata information of each node in the to-be-restored cluster, the cluster structure is reconstructed, the backup set data is distributed to each node in the target cluster, and physical recovery is performed in parallel;

[0130] Based on each node in the restored target cluster, a cluster management tool is used to associate cluster relationship and perform cluster registration operation, and a new cluster corresponding to the to-be-restored cluster after recovery is obtained.

[0131] In one embodiment, the processor further implements the steps of the database cluster recovery method in the above-mentioned other embodiments when executing the computer program.

[0132] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0133] By extracting cluster configuration information and node state information of the to-be-restored cluster, metadata information of each node in the to-be-restored cluster is obtained; the metadata information at least includes node role and connection parameter of the corresponding node, and node running state; the node role includes master node and slave node;

[0134] Backup set data of the to-be-restored cluster matched with the cluster recovery requirement information is obtained; the backup set data is database business data written by the master node in the to-be-restored cluster;

[0135] According to the overall running state of the target cluster, the node role layout of the target cluster is determined; the target cluster is used to restore the to-be-restored cluster which has failed across environments;

[0136] reconstructing a cluster structure according to the node role layout and the metadata information of each node in the to-be-restored cluster, distributing the backup set data to each node in the target cluster, and performing physical restoration in parallel;

[0137] Based on each node in the restored target cluster, a cluster management tool is used to associate cluster relationships and perform cluster registration operations to obtain a new cluster corresponding to the to-be-restored cluster after restoration.

[0138] In one embodiment, the computer program, when executed by a processor, also implements the steps of the database cluster restoration method in the other embodiments described above.

[0139] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the following steps:

[0140] By extracting cluster configuration information and node state information of the to-be-restored cluster, metadata information of each node in the to-be-restored cluster is obtained; the metadata information at least includes node roles and connection parameters of the corresponding nodes, and node running states; the node roles include master nodes and slave nodes;

[0141] Backup set data of the to-be-restored cluster matching the cluster restoration requirement information is obtained; the backup set data is database business data written by the master nodes in the to-be-restored cluster;

[0142] According to the overall running state of the target cluster, a node role layout of the target cluster is determined; the target cluster is used to restore the to-be-restored cluster that has failed across environments;

[0143] According to the node role layout and the metadata information of each node in the to-be-restored cluster, a cluster structure is reconstructed, the backup set data is distributed to each node in the target cluster, and physical restoration is performed in parallel;

[0144] Based on each node in the restored target cluster, a cluster management tool is used to associate cluster relationships and perform cluster registration operations to obtain a new cluster corresponding to the to-be-restored cluster after restoration.

[0145] In one embodiment, the computer program, when executed by a processor, also implements the steps of the database cluster restoration method in the other embodiments described above.

[0146] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0148] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0149] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A database cluster recovery method, characterized in that, The method includes: By extracting the cluster configuration information and node status information of the cluster to be recovered, the metadata information of each node in the cluster to be recovered is obtained; the metadata information includes at least the node role and connection parameters of the corresponding node, as well as the node running status; the node role includes master node and slave node; Obtain backup set data of the cluster to be restored that matches the cluster recovery requirements; the backup set data is database business data written by the master node in the cluster to be restored. Based on the overall operating status of the target cluster, the node role layout of the target cluster is determined; the target cluster is used to recover the cluster to be recovered from failure across environments; The cluster structure is reconstructed according to the node role layout and the metadata information of each node in the cluster to be restored. The backup set data is distributed to each node in the target cluster, and physical recovery is performed in parallel. Based on the nodes in the restored target cluster, cluster management tools are used to associate cluster relationships and perform cluster registration operations to obtain the new restored cluster corresponding to the cluster to be restored.

2. The method according to claim 1, characterized in that, The process of acquiring backup set data of the cluster to be restored that matches the cluster recovery requirements includes: Based on the cluster recovery requirement information, obtain the specified recovery time point; Query the backup set that matches the recovery time point in the backup server to obtain the backup set data of the cluster to be recovered.

3. The method according to claim 1, characterized in that, The step of determining the node role layout of the target cluster based on the overall operating status of the target cluster includes: Based on the overall operating status of the target cluster, it is determined that each node and its instance in the target cluster are operating normally; The node role layout is obtained based on the current master-slave node information of the target cluster.

4. The method according to claim 1, characterized in that, The step of determining the node role layout of the target cluster based on the overall operating status of the target cluster includes: Based on the overall operating status of the target cluster, it is determined that the target cluster is in an unavailable state; The node role layout is obtained based on the pre-extracted cluster metadata information of the target cluster.

5. The method according to claim 1, characterized in that, The step of distributing the backup set data to each node in the target cluster and performing physical recovery in parallel includes: The backup set data is distributed to each node in the target cluster; The recovery operation is performed simultaneously on each node in the target cluster, and the backup set data is used to overwrite the original data on each node in the target cluster.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Start the new cluster after recovery; Based on the operational health status of the newly restored cluster, a node role determination operation is performed to allocate master and slave nodes to the newly restored cluster.

7. A database cluster recovery device, characterized in that, The device includes: The metadata extraction module is used to obtain the metadata information of each node in the cluster to be recovered by extracting the cluster configuration information and node status information of the cluster to be recovered; the metadata information includes at least the node role and connection parameters of the corresponding node, as well as the node running status; the node role includes master node and slave node; The backup set acquisition module is used to acquire backup set data of the cluster to be recovered that matches the cluster recovery requirement information; the backup set data is database business data written by the master node in the cluster to be recovered. The node role layout determination module is used to determine the node role layout of the target cluster based on the overall operating status of the target cluster; the target cluster is used to recover the cluster to be recovered from a failure across environments; The parallel recovery module is used to reconstruct the cluster structure according to the node role layout and the metadata information of each node in the cluster to be recovered, distribute the backup set data to each node in the target cluster, and perform physical recovery in parallel. The cluster registration module is used to associate cluster relationships and perform cluster registration operations based on each node in the recovered target cluster using cluster management tools, thereby obtaining the new recovered cluster corresponding to the cluster to be recovered.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images