Cluster snapshot-based disaster recovery method and device for distributed database system and medium

Abstract

The application relates to a cluster snapshot-based distributed database system disaster recovery method, equipment and medium. The method comprises the following steps: when a distributed database cluster is normally operated, generating a cluster snapshot based on a storage-computing separation architecture, the cluster snapshot comprising a metadata snapshot and a data snapshot, uploading the metadata snapshot to an object storage, and recording a data version corresponding to the snapshot; during normal operation of the distributed database cluster, continuously performing a cluster snapshot generation operation, forming a snapshot sequence comprising multiple snapshot versions, and performing a retention and deletion operation in the object storage; when the distributed database cluster fails, performing a recovery process according to the cluster snapshot in the object storage; and after the recovery of the distributed database cluster is completed, updating node address information of the distributed database cluster according to node configuration in a newly deployed environment. The application has the effect of improving the system disaster recovery capability.

Description

Technical Field

[0001] This application relates to the technical field of electronic digital data processing, and in particular to a disaster recovery method, device and medium for a distributed database system based on cluster snapshots. Background Technology

[0002] Currently, distributed database systems are widely deployed in large-scale data processing and cloud computing architectures. As business demands for data reliability and continuous availability continue to increase, the disaster recovery capability of distributed databases has become a key indicator in system design, especially in scenarios where node failures, cluster unavailability, or regional outages may occur.

[0003] Traditional disaster recovery solutions often employ a multi-cluster deployment and synchronization approach, deploying primary and backup clusters in multiple availability zones and ensuring disaster recovery failover through real-time or near real-time data replication. However, such solutions are costly, increasing cluster resource and management overhead, and placing higher demands on network bandwidth and synchronization consistency mechanisms.

[0004] Currently, the industry widely adopts backup and recovery-based disaster recovery, which requires that when the distributed database cluster is working normally, the data in the distributed database system be backed up to a reliable object storage regularly. When the distributed database cluster crashes, the data backed up on the object storage can be restored to the new distributed database cluster, thus achieving disaster recovery of the distributed database cluster. However, traditional backup and recovery mechanisms have problems such as low efficiency, high cost, and significant impact on system operation.

[0005] The existing technical solutions mentioned above have the following drawbacks: existing disaster recovery mechanisms mostly rely on multi-cluster deployment or periodic backups, which increases system resource consumption and leads to low recovery efficiency and complex switching processes. They are difficult to meet the comprehensive requirements of modern distributed databases in terms of high availability, low cost and fast recovery, and therefore there is room for improvement. Summary of the Invention

[0006] To improve system disaster recovery capabilities, this application provides a method, device, and medium for disaster recovery of a distributed database system based on cluster snapshots.

[0007] The above-mentioned objective of this application is achieved through the following technical solution:

[0008] A disaster recovery method for a distributed database system based on cluster snapshots, the method comprising:

[0009] When the distributed database cluster is running normally, based on the storage-compute separation architecture, a cluster snapshot is generated. The cluster snapshot includes a metadata snapshot and a data snapshot. The metadata snapshot is uploaded to the object storage, and the data version corresponding to the snapshot is recorded.

[0010] During the normal operation of the distributed database cluster, cluster snapshot generation operations are continuously performed to form a snapshot sequence containing multiple snapshot versions, and retention and deletion operations are performed in the object storage;

[0011] When the distributed database cluster fails, a recovery process is executed based on the cluster snapshot in the object storage;

[0012] After the distributed database cluster is restored, the node address information of the distributed database cluster is updated according to the node configuration in the new deployment environment.

[0013] By adopting the above technical solutions, and generating cluster snapshots based on a storage-compute separation architecture during normal operation of the distributed database cluster, stateless operation of compute nodes, unified data storage, and snapshot persistence can be achieved, thereby enhancing system scalability and data consistency. By dividing cluster snapshots into metadata snapshots and data snapshots, and uploading the metadata snapshots to object storage while recording data versions, complete preservation and traceability of the cluster state can be achieved, thus providing a structural closed loop for subsequent disaster recovery. By continuously generating snapshots during cluster operation, forming a snapshot version sequence, and performing retention and deletion operations in object storage, the number of snapshots can be controlled, storage costs can be reduced, thereby ensuring the controllability of recovery points and resource utilization. After a cluster failure, the recovery process is executed based on the snapshots saved in object storage, and the node address information is updated according to the new deployment environment after recovery, rapid recovery and adaptation across environments and regions can be achieved, thereby significantly improving system disaster recovery capabilities and operational flexibility.

[0014] In one example, this application can be further configured such that: the generation of cluster snapshots based on the storage-compute separation architecture specifically includes:

[0015] In the metadata management node deployed in the distributed database cluster, a full metadata snapshot is generated based on the distributed consensus protocol. The full metadata snapshot includes data sharding information, data version identifier, scheduling control information, and transaction index and status information for log replay. The full metadata snapshot is used as the metadata part of the cluster snapshot.

[0016] In the data computing nodes of the distributed database cluster, the multi-version data files generated during the operation of the distributed database cluster are written to the object storage for storage, forming the data portion of the cluster snapshot;

[0017] Based on the full metadata snapshot, the corresponding target data version set is recorded, and the target data version set is used as the range boundary of the data snapshot in the cluster snapshot.

[0018] By adopting the above technical solutions, generating a full metadata snapshot based on a distributed consensus protocol in the metadata management node ensures strong consistency and global coordination capabilities in the snapshot state, thereby guaranteeing scheduling and transaction consistency during recovery scenarios. By limiting the content of the full metadata snapshot to data shard information, data version identifiers, scheduling control information, and transaction indexes and status information for log replay, key information of the cluster control plane can be completely recorded, thus supporting accurate restoration of the running state. By writing multiple versions of data files to object storage in the compute nodes, and by recording the target data version set based on the full metadata snapshot as the boundary of the data snapshot range, a mapping relationship between data and metadata can be established, thereby ensuring version matching and data consistency during the recovery process.

[0019] In one example, this application can be further configured such that generating a full metadata snapshot based on a distributed consensus protocol specifically includes:

[0020] In the metadata management node, a full metadata snapshot is generated by executing a replica state synchronization mechanism based on the distributed consensus protocol, wherein the distributed consensus protocol includes the Raft consensus algorithm or the Paxos consensus algorithm.

[0021] When the preset triggering conditions for snapshot generation are met, a snapshot file of the current consistency state is generated. The snapshot file serves as the full metadata snapshot. The preset triggering conditions include the snapshot period reaching a preset time interval.

[0022] By adopting the above technical solution, a full metadata snapshot is generated by executing a replica state synchronization mechanism based on a distributed consensus protocol in the metadata management node. This enables the acquisition of consistent snapshots without interrupting cluster operation, thereby ensuring the correctness and recoverability of snapshot data. By setting preset trigger conditions for snapshot generation, including the snapshot cycle at preset time intervals, the cluster state can be periodically solidified, thereby avoiding the risk of snapshot loss due to sudden failures and improving the timeliness and coverage integrity of snapshots.

[0023] In one example, this application can be further configured as follows: the continuous execution of cluster snapshot generation operations to form a snapshot sequence containing multiple snapshot versions, and the execution of retention and deletion operations in the object storage, specifically including:

[0024] A cluster snapshot generation task is triggered according to the preset time interval;

[0025] After completing a new round of cluster snapshot generation, the number of existing snapshot copies in the object storage is determined based on a preset retention strategy.

[0026] When the number of snapshot replicas does not exceed the preset retention limit, all existing cluster snapshots are retained to form a snapshot sequence.

[0027] When the number of snapshot replicas exceeds the preset retention limit, the earliest generated cluster snapshot is deleted in the order of snapshot creation time.

[0028] By adopting the above technical solutions, and by triggering cluster snapshot generation tasks according to preset time intervals, the snapshot generation operation can be ensured to be periodic and systematic, thereby improving the coverage density of recovery points. By determining the number of snapshot replicas based on the retention policy after each round of snapshot generation, dynamic control of the number of snapshots in object storage can be achieved, thereby reducing storage resource consumption. By retaining all snapshots when the number of replicas is within the limit, and deleting the earliest snapshot by creation time when the limit is exceeded, the integrity of snapshot retention and the efficiency of historical space reclamation can be balanced, thereby ensuring that the system has a continuously available recovery window while controlling costs.

[0029] In one example, this application can be further configured such that: the recovery process is executed based on the cluster snapshot in the object storage, specifically including:

[0030] Load the latest retained metadata snapshot from the current snapshot sequence from the object storage, and restore the metadata state of the distributed database cluster based on the latest retained metadata snapshot to obtain the restored metadata information;

[0031] Based on the latest transaction index and status information recorded in the latest retained metadata snapshot, the incremental log is selectively replayed;

[0032] After restoring the metadata status of the distributed database cluster, the corresponding version of the data file in the object storage is accessed based on the restored metadata information to restore the normal operation capability of the distributed database cluster.

[0033] By adopting the above technical solutions, the cluster control plane information can be quickly restored by loading the latest retained metadata snapshot from the current snapshot sequence in the object storage to restore the metadata state, thus laying the foundation for subsequent data scheduling and access. By selectively replaying incremental logs based on the transaction indexes and status information recorded in the metadata snapshot, the recovery time can be effectively shortened and unnecessary log processing can be reduced, thereby improving the overall recovery efficiency. By accessing the corresponding version of the data file in the object storage according to the restored metadata information, the consistency between the data plane and the control plane can be ensured, thereby achieving complete, available, and consistent recovery of the distributed database cluster.

[0034] In one example, this application can be further configured as follows: updating the node address information of the distributed database cluster according to the node configuration in the new deployment environment specifically includes:

[0035] Based on the instance identifiers and corresponding network connection parameters of each node in the newly deployed environment, a node address mapping table is generated for the current environment, wherein each node includes compute nodes and metadata management nodes.

[0036] Based on the node address mapping table, the original node identifier in the latest retained metadata snapshot is replaced to obtain the updated node address information;

[0037] The updated node address information is synchronized to each metadata management node and data computing node in the newly deployed environment.

[0038] By adopting the above technical solutions, a node address mapping table can be generated based on the instance identifier and network connection parameters of nodes in the new deployment environment. This enables node mapping adaptation between different deployment environments, thereby supporting cross-environment disaster recovery and migration. By replacing the original node identifier in the metadata snapshot and generating updated node address information, scheduling anomalies or network disconnections caused by environmental differences can be avoided, thus ensuring normal communication and operation of each node after cluster recovery. By synchronizing the updated node address information to each metadata management node and data computing node, the consistency maintenance of cluster configuration after address changes can be completed, thereby improving automated recovery and deployment capabilities.

[0039] In one example, this application can be further configured to: synchronize the updated node address information to each metadata management node and data computing node in the new deployment environment, specifically including:

[0040] During the synchronization of the updated node address information, if it is detected that any target node has failed to receive the updated node address information, the synchronization operation will be re-initiated based on a preset retry mechanism, which includes a maximum number of retry attempts and an interval time strategy.

[0041] If node address synchronization is not completed after reaching the maximum number of retries, the status information of the corresponding failed node is recorded, and a disaster recovery alarm is triggered.

[0042] By adopting the above technical solutions, introducing a failure detection mechanism during node address information synchronization, and re-initiating the synchronization operation based on a preset retry mechanism, the success rate of node information update operations can be improved, thereby enhancing the reliability of the system in complex network environments. By setting an upper limit on the number of retry attempts and an interval time strategy, synchronization efficiency and system load can be balanced, thereby avoiding the impact of the synchronization process on cluster operation. By recording the status and triggering disaster recovery alarms when the maximum number of retries still fails, it can help operation and maintenance personnel to identify abnormal nodes in a timely manner and take remedial measures, thereby improving the observability and controllability of cluster disaster recovery operations.

[0043] In one example, this application can be further configured as follows: the disaster recovery method for a distributed database system based on cluster snapshots also includes:

[0044] The cluster snapshots in the object storage are periodically synchronized to the target region object storage service deployed in a different location, and the snapshot storage structure and naming path in the target region object storage are kept consistent with those in the object storage. The synchronization operation of the cluster snapshots includes full replication mode and incremental replication mode.

[0045] By adopting the above technical solution, and periodically synchronizing cluster snapshots in object storage to the target region object storage deployed in a different location, while maintaining the same snapshot structure and naming path as in the original object storage, a unified multi-region snapshot view can be built, thereby achieving disaster recovery capabilities across availability zones or regions. By supporting both full replication and incremental replication synchronization modes, the snapshot structure can be quickly built during the initial deployment, and changes can be efficiently synchronized in subsequent cycles, thereby reducing network transmission pressure and synchronization time, and improving data consistency and recovery efficiency.

[0046] The above-mentioned objective 2 of this application is achieved through the following technical solution:

[0047] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described disaster recovery method for a distributed database system based on cluster snapshots.

[0048] The above-mentioned objective three of this application is achieved through the following technical solution:

[0049] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described disaster recovery method for a distributed database system based on cluster snapshots.

[0050] In summary, this application includes the following beneficial technical effects:

[0051] 1. By generating cluster snapshots based on the storage-compute separation architecture during normal operation of the distributed database cluster, stateless operation of compute nodes, unified data storage, and snapshot persistence can be achieved, thereby enhancing system scalability and data consistency assurance. By dividing the cluster snapshot into metadata snapshots and data snapshots, and uploading the metadata snapshots to object storage while recording the data version, the complete preservation and traceability of the cluster state can be achieved, thereby providing a structural closed loop for subsequent disaster recovery.

[0052] 2. By continuously generating snapshots during cluster operation to form a snapshot version sequence, and performing retention and deletion operations in object storage, the number of snapshots can be controlled and storage costs reduced, thereby ensuring the controllability of recovery points and resource utilization. After a cluster failure, the recovery process is executed based on the snapshots saved in object storage, and the node address information is updated according to the new deployment environment after recovery is completed. This enables rapid recovery and adaptation across environments and regions, thereby significantly improving the system's disaster recovery capabilities and operational flexibility. Attached Figure Description

[0053] Figure 1 This is a flowchart of a disaster recovery method for a distributed database system based on cluster snapshots in one embodiment of this application;

[0054] Figure 2 This is a flowchart illustrating the implementation of step S10 in a disaster recovery method for a distributed database system based on cluster snapshots, according to one embodiment of this application.

[0055] Figure 3 This is a schematic diagram of a device according to one embodiment of this application. Detailed Implementation

[0056] The present application will be further described in detail below with reference to the accompanying drawings.

[0057] In one embodiment, such as Figure 1 As shown, this application discloses a disaster recovery method for a distributed database system based on cluster snapshots, which specifically includes the following steps:

[0058] S10: When the distributed database cluster is running normally, based on the storage-compute separation architecture, a cluster snapshot is generated. The cluster snapshot includes a metadata snapshot and a data snapshot. The metadata snapshot is uploaded to the object storage, and the data version corresponding to the snapshot is recorded.

[0059] Specifically, during normal operation of the distributed database cluster, metadata content such as the current coordination and scheduling status, sharding information, and data version mapping is periodically extracted from the metadata management node of the distributed database cluster to construct metadata snapshots. During the operation of the data computing node, multiple versions of data files are written to object storage in real time using a storage-compute separation architecture to form data snapshots. The generated metadata snapshots are uploaded to the snapshot storage area specified in the object storage path through an interface, and at the same time, the corresponding data version set is bound with a unique snapshot identifier, forming a snapshot folder structure with a unified structure, clear path, and traceable data and metadata. The metadata snapshot file is uploaded to the specified object storage path, for example, creating the directory / snapshot / 20251231_0100 / meta.snapshot, and registering the version number, time period, and data version range of the snapshot in the database snapshot index record module to ensure fast retrieval or backtracking as needed in the future.

[0060] S20: During the normal operation of the distributed database cluster, continuously execute cluster snapshot generation operations to form a snapshot sequence containing multiple snapshot versions, and perform retention and deletion operations in object storage.

[0061] Specifically, during the normal operation of the distributed database cluster, a periodic snapshot task trigger is started. Every set time interval (e.g., every minute), a snapshot generation process is triggered. The snapshot generation process is executed non-blockingly in the background, including packaging the current metadata state snapshot and marking the data snapshot boundaries. After generation, the snapshots are uniformly saved to the snapshot root directory of the object storage and named in chronological order, such as / snapshot / 20251231_0100, / snapshot / 20251231_0105, etc. Then, the snapshot retention policy component is called to scan the current snapshot sequence. When the number of snapshots exceeds the configured maximum retention number (e.g., 10), the old snapshot directory and its subordinate snapshot contents, including metadata snapshot files and data snapshot index records, are deleted sequentially according to the earliest generation time. This ensures controllable use of object storage space and provides a sliding window-like snapshot retention effect. For example, under a policy setting to retain a maximum of 10 snapshots, if there are already 10 snapshots, adding the 11th snapshot will automatically delete the earliest snapshot _202512290101.

[0062] S30: When a distributed database cluster fails, a recovery process is executed based on the cluster snapshot in object storage.

[0063] Specifically, upon detecting a cluster crash or unavailability, the latest available cluster snapshot is selected from object storage for recovery. First, the metadata snapshot file contained in the latest available cluster snapshot is loaded, and the structured data such as sharding information, version mapping, and transaction indexes are parsed out to rebuild the memory state model of the metadata management node. At the same time, the corresponding data file path is located according to the recorded target data version set, and the data computing node is authorized to access the relevant data content in object storage to complete the computation recovery. Throughout the process, there is no need to copy the data content back to the local disk. Instead, zero-copy startup is achieved through lazy loading and lazy initialization. For example, when using snapshot SN_20251230_0020 to recover the cluster, it is only necessary to download the metadata snapshot to the master node and load it. After the data node starts, it directly accesses files such as " / tableA / shard1 / v125.parquet" under the OSS path according to the metadata mapping to complete the recovery.

[0064] S40: After the distributed database cluster has recovered, update the node address information of the distributed database cluster according to the node configuration in the new deployment environment.

[0065] Specifically, after the initial state recovery of the cluster is completed, the current instance identifier and network connection parameters of each node are parsed from the node registration information in the new deployment environment. An address mapping table reflecting the relationship between metadata nodes and data computing nodes in the current environment is constructed. Then, this mapping table is compared and replaced item by item with the old addresses recorded in the cluster snapshot to ensure that the addresses of each node referenced in the metadata are consistent with the actual accessible addresses in the new environment. After the replacement is completed, the new addresses are synchronously distributed to each node for initializing connection relationships and load distribution. For example, if the original computing node identifier recorded in the snapshot is C1:10.0.0.8, and the C1 node is redeployed to 192.168.3.101:8080 in the new environment, the address mapping table will replace the original entries with the new addresses and write them to the metadata recovery buffer to complete the unified update.

[0066] By adopting the above technical solutions, and generating cluster snapshots based on a storage-compute separation architecture during normal operation of the distributed database cluster, stateless operation of compute nodes, unified data storage, and snapshot persistence can be achieved, thereby enhancing system scalability and data consistency. By dividing cluster snapshots into metadata snapshots and data snapshots, and uploading the metadata snapshots to object storage while recording data versions, complete preservation and traceability of the cluster state can be achieved, thus providing a structural closed loop for subsequent disaster recovery. By continuously generating snapshots during cluster operation, forming a snapshot version sequence, and performing retention and deletion operations in object storage, the number of snapshots can be controlled, storage costs can be reduced, thereby ensuring the controllability of recovery points and resource utilization. After a cluster failure, the recovery process is executed based on the snapshots saved in object storage, and the node address information is updated according to the new deployment environment after recovery, rapid recovery and adaptation across environments and regions can be achieved, thereby significantly improving system disaster recovery capabilities and operational flexibility.

[0067] In one embodiment, such as Figure 2 As shown, in step S10, which is based on the storage-compute separation architecture, a cluster snapshot is generated, specifically including:

[0068] S11: In the metadata management node deployed in the distributed database cluster, a full metadata snapshot is generated based on the distributed consensus protocol. The full metadata snapshot includes data sharding information, data version identifier, scheduling control information, and transaction index and status information for log replay. The full metadata snapshot is used as the metadata part of the cluster snapshot.

[0069] Specifically, in the metadata management node, when the snapshot triggering conditions are detected internally, the cluster consistency view held by the current master node is obtained through the configured distributed consensus protocol. The mapping relationship between all logical shards and physical nodes in the view is extracted, and together with the latest data version number corresponding to each shard, the node load status and replication strategy defined in the current cluster scheduling module, and the committed transaction number and status snapshot extracted from the distributed transaction log, they are encapsulated into a structured data block. In this process, the version boundary of the full snapshot is marked by a sequence number and the time when the status takes effect is recorded. For example, in a typical snapshot generation, the metadata includes a primary key table A with 6 shards distributed on 3 compute nodes, each with a corresponding data version of v1 to v6. At the same time, the scheduling information indicates that the replication distribution strategy is "3 replicas across AZ balanced", and the maximum number covered by the transaction index is T5248. Finally, this information is compressed and encoded into a snapshot file, written to the metadata directory, and submitted as the metadata part of the cluster snapshot.

[0070] S12: In the data computing nodes of the distributed database cluster, the multi-version data files generated during the operation of the distributed database cluster are written to object storage for storage, forming the data portion of the cluster snapshot.

[0071] Specifically, during the data write, update, or delete operation process in the distributed database cluster, the data computing node encapsulates the changed data content through an internal version control mechanism and generates a unique data version identifier for each data change. The corresponding data version file is written to the specified path directory in the object storage in an immutable format to complete remote persistence. At the same time, the shard identifier, version number, and logical path index corresponding to each version file are recorded in the metadata.

[0072] In this embodiment, to improve access efficiency during the data reading phase, after writing multiple versions of data files to object storage, the data computing node can selectively retain a portion of persistent cached data locally. This cached data is typically the most frequently used hot versions or transaction-related data blocks. By maintaining a lightweight local file mapping and reference index, the node prioritizes matching the local cache during subsequent query or recovery operations. If the local cache hits, the required data file is read directly without accessing the remote object storage, thereby significantly reducing object storage I / O latency and network transmission bandwidth consumption, and improving read efficiency. For example, when performing a read operation on shard #4 version v301 of table A, if this version is already cached in the local SSD of node C2, the v301.parquet file is read directly to complete the query response. If the cache misses, the node falls back to the object storage path to load remote data and triggers an asynchronous cache update mechanism, thereby achieving overall optimization of the performance of tiered access to hot and cold data and snapshot data read.

[0073] S13: Based on the full metadata snapshot, record the corresponding target data version set, and use the target data version set as the range boundary of the data snapshot in the cluster snapshot.

[0074] Specifically, after generating a full metadata snapshot, all shards and their corresponding data version identifiers contained in the snapshot are parsed. These data version numbers are aggregated to form the target data version set for the current cluster snapshot. This set is used to define the boundary range of the data snapshot content. If an update occurs during cluster operation that causes some data versions in object storage to be later than the version number identified in this set, they will not be included in the current snapshot content but will only be referenced in subsequent snapshot generation. For example, in snapshot number SN220, the three shard versions of metadata snapshot record table C are v13, v18, and v21, respectively. Then the target data version set is {v13, v18, v21}. If v22 or higher versions already exist in object storage, they will not be included in the data snapshot range corresponding to this snapshot. This ensures the consistent reference relationship between metadata and data and provides a precise backtracking boundary for recovery operations.

[0075] To further improve the frequency and efficiency of snapshot generation, an incremental logging mechanism can be initiated after the initial generation of a full metadata snapshot. After each cluster state change, an incremental change log fragment is constructed by parsing the metadata state machine log, and this incremental log file is periodically uploaded to the log path directory in object storage. The log content includes operation information such as shard reassignment, data version updates, and transaction commit state changes that have occurred since the last snapshot. When executing the recovery process, the most recent full metadata snapshot is loaded first as the baseline state, and then the incremental log sequence matching the snapshot is replayed in sequence to reconstruct the latest consistent state close to the time of the crash. This mechanism can reduce the snapshot generation frequency from minutes to seconds, effectively reducing the RPO value and increasing the time density of snapshots.

[0076] In one embodiment, step S11, namely generating a full metadata snapshot based on the distributed consensus protocol, specifically includes:

[0077] S111: In the metadata management node, a full metadata snapshot is generated by executing a replica state synchronization mechanism based on a distributed consensus protocol. The distributed consensus protocol includes the Raft consensus algorithm or the Paxos consensus algorithm.

[0078] Specifically, when executing the replica state synchronization mechanism, the consistency state range of each node replica is confirmed by broadcasting the current node state summary to all participating metadata replica nodes and collecting responses. After the state is consistent, the snapshot construction process is initiated, and the metadata structure information in the current node is extracted into the memory buffer. The metadata structure includes the sharding metadata of the data table, the version number mapping of each shard, the data node distribution mapping, and the transaction context summary in progress. At the same time, the state data in the buffer is packaged into an intermediate snapshot file through a serialization strategy, and other replicas are notified within the synchronization window to record the current snapshot number for replay confirmation. For example, in the Raft protocol, the Leader node initiates the InstallSnapshot request. After successfully completing the snapshot synchronization, the snapshot number is recorded in the local metadata storage component and marked as "recoverable state" for subsequent snapshot registration and replay operations.

[0079] S112: When the preset triggering conditions for snapshot generation are met, a snapshot file of the current consistency state is generated. The snapshot file serves as a full metadata snapshot. The preset triggering conditions include the snapshot period reaching a preset time interval.

[0080] Specifically, when the background snapshot task detects that the running time since the last snapshot generation exceeds the set trigger cycle threshold, it automatically starts a snapshot generation operation. By scheduling the snapshot controller bound in the snapshot task thread, it triggers a full sealing operation of the current node's internal state. During the execution process, it first freezes the variable field access interface of the current metadata to avoid data races. Then, it parses the metadata structure of the current stable state from the committed transaction logs and scheduling queue, and writes it to the temporary snapshot file path using a unified format. After the snapshot file is generated, it will synchronously record its corresponding transaction index position and system status stamp, and update the local metadata snapshot sequence number and its corresponding validity period mark. For example, it is set to retain 10 snapshot versions for sliding rollback recovery. Finally, after satisfying the format verification and status verification, the snapshot file is submitted to the local path to wait for the upload command to be triggered.

[0081] In one embodiment, step S20 involves continuously performing cluster snapshot generation operations to form a snapshot sequence containing multiple snapshot versions, and performing retention and deletion operations in object storage, specifically including:

[0082] S21: Trigger a cluster snapshot generation task according to a preset time interval.

[0083] Specifically, during the operation of the distributed database cluster, the interval between the current time and the timestamp of the last successfully created snapshot is continuously monitored. When the interval exceeds a preset threshold, a new round of cluster snapshot generation process is automatically triggered. Before triggering, a fast consistency check is performed on the current running status of the cluster to ensure the stability and consistency of the snapshot generation. The snapshot task is then distributed asynchronously to the metadata management node and the object storage upload module to work together. For example, if the snapshot period is set to 60 seconds, then when the last snapshot completion time is 12:00:00 and the current time reaches 12:01:01, a new round of snapshot task is triggered, and the task number is incremented to SN_20251230_0016 for subsequent tracking and snapshot sequence sorting.

[0084] In this application, the "preset time interval" refers to the time period threshold used to control the periodic generation of cluster snapshots. This time interval is used both as the trigger condition for determining whether to generate metadata snapshots in S112 and as the periodic scheduling logic in S21 to start the complete cluster snapshot generation task. Both share the same set of time parameter configurations to ensure that the generation of cluster snapshots has controllable time accuracy and frequency. For example, the time interval can be set to 60 seconds, 300 seconds, or other period lengths that meet the requirements of business disaster recovery. The system continuously determines whether the timing conditions for creating a snapshot are met based on this time interval, thereby achieving consistency and controllable timing of snapshot creation triggering.

[0085] S22: After completing the generation of a new round of cluster snapshots, the number of existing snapshot replicas in object storage is determined based on the preset retention policy.

[0086] Specifically, after completing the current snapshot task and successfully uploading the new snapshot file to object storage, all snapshot metadata under the snapshot storage path is scanned and parsed. The number of currently retained snapshot copies is counted by reading the snapshot naming rules or the attached metadata description file. This count is compared with the preset maximum retention limit. At the same time, the creation time and number of the newly added snapshot in this round are recorded as the tail index of the snapshot sequence. For example, when there are 10 historical snapshot files under the object storage path and the current retention limit is set to 12, the count result returns 10. The subsequent judgment module confirms that the current state has not exceeded the limit and skips the cleanup process to enter the next round of snapshot scheduling preparation.

[0087] S23: When the number of snapshot replicas does not exceed the preset retention limit, all existing cluster snapshots are retained to form a snapshot sequence.

[0088] Specifically, if the number of snapshots does not exceed the retention limit, the newly generated snapshot copy is directly appended to the end of the snapshot sequence. Meta-information such as the snapshot number, creation time, referenced data version range, and status label is registered in the snapshot index table, forming a snapshot queue arranged in ascending order of time. Each snapshot in this snapshot queue has independent recovery capabilities, and there may be some overlap between data version ranges. A sliding window snapshot maintenance mechanism is implemented through queue head and tail pointers. For example, when the current snapshot sequence number is SN_20251230_0010 to SN_20251230_0020, after the new snapshot SN_20251230_0021 is added, the sequence is updated to 0010~0021, which facilitates rollback recovery by sequence number or cross-cluster snapshot cloning operations.

[0089] S24: When the number of snapshot replicas exceeds the preset retention limit, delete the earliest generated cluster snapshot in the order of snapshot creation time.

[0090] Specifically, after detecting that the number of snapshot copies has exceeded the preset limit, the snapshot cleanup scheduler scans the snapshot index table and extracts the earliest generated snapshot record in the sequence. By comparing the snapshot creation time or sequence number, the target snapshot entry to be deleted is determined. Then, its metadata snapshot file and reference mark in object storage are deleted in turn, and the associated data version protection flag is cleared to free up storage space. At the same time, the index information of the snapshot is removed from the snapshot queue to maintain the continuity of the snapshot sequence. For example, when the current snapshot sequence is SN_20251230_0008 to SN_20251230_0020 and the maximum retention limit is 12, the cleanup process is triggered after adding SN_20251230_0021, automatically deleting the earliest snapshot number 0008 and updating the snapshot header pointer to point to 0009.

[0091] After a snapshot's lifecycle ends and it is removed from the snapshot sequence, the set of data versions originally referenced by that snapshot will automatically lose its version protection status and will no longer participate in the object storage GC task's whitelist. Subsequently, the cleanup scheduling module will perform version liveness checks on it during the next round of object storage cleanup. If the data version is not referenced by any other snapshot or active transaction, it will be marked as a cleanupable object and reclaimed during the GC cycle. This achieves the protection binding and decoupling logic between snapshots and data. For example, after snapshot SN_20251230_0023 is deleted, its associated data versions v201, v202, v203, etc., will be marked as non-referenced and cleared and released in the subsequent GC process.

[0092] In one embodiment, step S30, namely, performing a recovery process based on the cluster snapshot in object storage, specifically includes:

[0093] S31: Load the latest retained metadata snapshot from the current snapshot sequence from the object storage, restore the metadata state of the distributed database cluster based on the latest retained metadata snapshot, and obtain the restored metadata information.

[0094] Specifically, after completing the snapshot sequence index scan, the latest timestamp or number of the metadata snapshot identifier in the current snapshot sequence is extracted, and the corresponding snapshot file content is loaded from the object storage path into the memory buffer. By parsing the data shard structure, node mapping relationship, data version information and transaction control fields sealed in the snapshot, the metadata state model of the distributed database cluster is reconstructed. This model is loaded into the metadata control kernel of the master node as the starting point of the current cluster state. At the same time, the metadata version number and metadata transaction snapshot stamp are updated to ensure version consistency. For example, when the loaded snapshot number is SN_20251230_0023, its content includes the distribution mapping of 9 shards of table A and table B, the corresponding data version {v78,v103,v222} and scheduling control fields. After parsing, the metadata information of table A shard #1 allocated to node C1 and the current version is v78 can be reconstructed for reference in the subsequent recovery process.

[0095] S32: Selectively replay incremental logs based on the latest transaction indexes and status information recorded in the latest retained metadata snapshot.

[0096] Specifically, after loading the metadata snapshot, the transaction consistency boundary at the time of snapshot generation is determined based on the transaction index number and status stamp recorded in the snapshot. The incremental log sequence file after the number is retrieved from the object storage. The log content is parsed one by one to determine whether it belongs to a valid transaction that was successfully committed but not included in the snapshot. If the condition is met, these logs are replayed in sequence, and the newly added metadata information is merged into the recovery status to complete the state changes within the time window between snapshot creation and recovery execution. The entire process can be configured as exact replay or batch replay to balance recovery speed and consistency requirements. For example, when the maximum transaction index in the snapshot is T5900, and the log sequence records commit operations from T5901 to T5904, these 4 logs can be replayed to complete operations such as creating table C and expanding table A sharding, so that the metadata status after the final recovery is logically consistent with that before the crash.

[0097] S33: After completing the metadata state recovery of the distributed database cluster, access the corresponding version of the data file in the object storage based on the recovered metadata information to restore the normal operation capability of the distributed database cluster.

[0098] Specifically, after the metadata state loading and incremental log replay are completed, the shard-node-version triplet mapping table in the metadata information is scanned. The access path index of each compute node to the target data file in object storage is constructed as needed. Delayed loading requests are initiated or the metadata preheating cache is read to start the data access task of the compute node. At the same time, the configuration policy determines whether to build the query index in advance or delay loading cold data content, thereby completing the functional recovery process of the data computing layer. In this process, it is not necessary to copy all the data to the local machine. Instead, the data source, mainly object storage, is used to perform on-demand calls. For example, after the recovery, shard #2 of table B is assigned to node C3 and the corresponding data version is v105. Node C3 can directly access the path " / data / tableB / shard2 / v105.parquet" to perform subsequent query, write or transaction processing operations, thereby ensuring that the cluster function is quickly restored with minimal data movement.

[0099] When performing recovery based on cluster snapshots, if there are higher version data files in object storage that are not part of the version set identified by the current snapshot, it will not affect the recovery and operation process of the current cluster. By default, the system only accesses and uses the data version set explicitly recorded in the target snapshot, while skipping other unreferenced data files to ensure data consistency and snapshot semantic accuracy. For example, when recovering table shards of version v103, if v104 and v105 exist in object storage at the same time, the recovery process will only load v103 for data initialization. This mechanism allows multiple versions to coexist in object storage without affecting the current recovery state. It can also be combined with cleanup strategies to perform redundant version reclamation after recovery to optimize storage utilization.

[0100] In one embodiment, step S40, which involves updating the node address information of the distributed database cluster according to the node configuration in the new deployment environment, specifically includes:

[0101] S41: Based on the instance identifiers and corresponding network connection parameters of each node in the new deployment environment, generate a node address mapping table for the current environment, where each node includes compute nodes and metadata management nodes.

[0102] Specifically, after the distributed database cluster is deployed to a new operating environment, the instance identifiers, container IDs, or hostnames of all compute nodes and metadata management nodes, as well as their bound network connection parameter information, including IP addresses, port numbers, communication protocol types, and reachability status, are collected through the environment resource management interface or the cluster initialization registry. Based on this collected information, a node address mapping table is constructed to establish a one-to-one correspondence between the logical node IDs in the original snapshot and the actual running nodes in the new environment. The field format in the mapping table is defined as {original node ID → current IP:port}. For example, if the address corresponding to the original metadata node M0 in the snapshot is 10.0.1.12:9010, and its container mapping address in the new environment is 192.168.3.45:9088, then the mapping entry {M0→192.168.3.45:9088} is generated for subsequent snapshot replacement operations.

[0103] S42: Based on the node address mapping table, replace the original node identifier in the latest retained metadata snapshot to obtain the updated node address information.

[0104] Specifically, after constructing the node address mapping table, the content of the latest retained metadata snapshot file is parsed to locate fields involving node identity, node communication address, shard allocation information, etc. Fields containing node identifiers from the old environment are replaced one by one. A key-value mapping method is used to replace the logical node number and original communication address appearing in the original snapshot with the corresponding new network address in the mapping table. During the replacement process, the original field structure is kept unchanged to ensure the semantic consistency of the snapshot. For example, in the snapshot, the record of shard #3 in table C is assigned to node D2 with the address 10.0.5.9:9021, while in the new environment D2 is assigned 172.30.11.52:9100. After replacement, the field is updated to shard #3 → D2@172.30.11.52:9100, thus completing the environmental adaptation of the snapshot semantics.

[0105] S43: Synchronize the updated node address information to each metadata management node and data computing node in the newly deployed environment.

[0106] Specifically, after all snapshot address fields have been replaced, the updated node address configuration file is broadcast to all newly deployed nodes through the cluster's internal control channel. The distribution process employs a master node-driven and multi-node parallel receiving mechanism. The metadata master node initiates the distribution task and attaches a version number signature. After receiving the configuration, the target node performs local verification and parsing processes, completes the mounting of the address table and refreshes the connection information, and updates the target address pool and gateway policy for remote connections in the node's internal registry. For example, when compute node C4 starts up and receives a configuration containing the new master node address M0@192.168.3.45:9088 in the address mapping table, it will immediately stop retrying the connection to the original address and instead initiate an initialization handshake to the new address, thereby achieving rapid adaptation and updating of the connection structure and communication path between nodes within the cluster.

[0107] In one embodiment, step S43, which involves synchronizing the updated node address information to each metadata management node and data computing node in the newly deployed environment, specifically includes:

[0108] S431: During the synchronization of updated node address information, if it is detected that any target node has failed to receive the updated node address information, the synchronization operation will be re-initiated based on the preset retry mechanism, which includes the upper limit of the number of retryes and the interval time strategy.

[0109] Specifically, during address synchronization, if a target node fails to respond to the address update confirmation within a specified time or experiences a communication anomaly leading to transmission failure, a retry mechanism is triggered. The scheduler re-executes the address synchronization task according to a preset strategy. By recording the instance identifier and failure round number of the failed node, the updated address configuration file is resent to the node when the next retry time arrives. At the same time, the interval between two retries is dynamically adjusted according to the set time strategy to avoid congestion or deadlock. The maximum number of attempts is limited throughout the retry process to prevent unlimited retransmission from consuming resources. For example, the maximum number of retries is configured to be 3, with the interval time increasing progressively to 10 seconds, 20 seconds, and 30 seconds. When node D5 is detected to have failed to respond to the initial synchronization request due to network congestion, the node initiates the address configuration transmission task to D5 again at 10 seconds, 30 seconds, and 60 seconds, and records the response status of each round to determine whether to enter the disaster recovery process.

[0110] S432: When the maximum number of retries is reached and node address synchronization is still not completed, record the status information of the corresponding failed node and trigger a disaster recovery alarm.

[0111] Specifically, if a node fails to receive a successful confirmation response for the address synchronization request from the target node after reaching the maximum number of retries, the system records the node's instance identifier, failure timestamp, role type, and recent communication anomaly information. This information is then aggregated in the cluster health monitoring module and written to the cluster status audit log. Simultaneously, a disaster recovery alarm mechanism is triggered to generate an alarm event. This event is then communicated to operations and maintenance personnel through pre-defined alarm channels such as email push, console alarms, or cluster console highlighting. The alarm content should include the node's disconnection time, the type of address synchronization failure, and suggested handling measures. For example, if node M1 fails to synchronize address information after three rounds of retries, the system will record "Node M1@192.168.3.100 synchronization failed, failure time 12:25:48, original address mapping invalid" and push it to the disaster recovery panel of the console. At the same time, an event log with the number ALERT#S432_001 is generated for review by the management backend.

[0112] In one embodiment, the disaster recovery method for a distributed database system based on cluster snapshots further includes:

[0113] S50: Periodically synchronize cluster snapshots in object storage to the target region object storage service deployed in a different location, and maintain the same snapshot storage structure and naming path in the target region object storage as in the object storage. The synchronization operation of cluster snapshots includes full replication mode and incremental replication mode.

[0114] Specifically, after the cluster snapshot generation task is completed, the snapshot synchronization scheduling module compares the snapshot index status under the current primary object storage path and the off-site backup area path. Based on the policy configuration, it decides whether to trigger a synchronization operation and selects between full or incremental replication mode. In full replication mode, the entire snapshot directory structure and all its file contents are extracted and copied completely. In incremental replication mode, based on the snapshot number and data version range of the metadata snapshot record, the newly added snapshot directory and data files since the last successful synchronization are identified and partially transferred. At the same time, the same directory hierarchy and file naming rules as the primary object storage are maintained in the target area object storage path to ensure path resolution and referencing during recovery. To maintain consistency, for example, when the synchronization task of snapshot SN_20251230_0026 is triggered at 12:00 on December 30, 2025, if the previously synchronized snapshot number is detected as 0024, the metadata snapshots corresponding to SN_20251230_0025 and 0026 and their referenced data versions {v223,v224} are copied in incremental mode to the target path "oss: / / backup-us-west / snapshots / ", ensuring that the snapshot structure in the off-site object storage is consistent with "oss: / / primary-cn / snapshots / ", thereby supporting cross-region disaster recovery, cluster cloning, or migration usage scenarios.

[0115] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0116] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a disaster recovery method for a distributed database system based on cluster snapshots.

[0117] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0118] When the distributed database cluster is running normally, based on the storage-compute separation architecture, a cluster snapshot is generated. The cluster snapshot includes a metadata snapshot and a data snapshot. The metadata snapshot is uploaded to the object storage, and the data version corresponding to the snapshot is recorded.

[0119] During the normal operation of the distributed database cluster, cluster snapshot generation operations are continuously executed to form a snapshot sequence containing multiple snapshot versions, and retention and deletion operations are performed in object storage;

[0120] When a distributed database cluster fails, a recovery process is executed based on a cluster snapshot in object storage.

[0121] After the distributed database cluster has recovered, update the node address information of the distributed database cluster according to the node configuration in the new deployment environment.

[0122] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0123] When the distributed database cluster is running normally, based on the storage-compute separation architecture, a cluster snapshot is generated. The cluster snapshot includes a metadata snapshot and a data snapshot. The metadata snapshot is uploaded to the object storage, and the data version corresponding to the snapshot is recorded.

[0124] During the normal operation of the distributed database cluster, cluster snapshot generation operations are continuously executed to form a snapshot sequence containing multiple snapshot versions, and retention and deletion operations are performed in object storage;

[0125] When a distributed database cluster fails, a recovery process is executed based on a cluster snapshot in object storage.

[0126] After the distributed database cluster has recovered, update the node address information of the distributed database cluster according to the node configuration in the new deployment environment.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0128] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0129] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A cluster snapshot-based distributed database system disaster recovery method, characterized in that, The disaster recovery method for distributed database systems based on cluster snapshots includes: When the distributed database cluster is running normally, based on the storage-compute separation architecture, a cluster snapshot is generated. The cluster snapshot includes a metadata snapshot and a data snapshot. The metadata snapshot is uploaded to the object storage, and the data version corresponding to the snapshot is recorded. During the normal operation of the distributed database cluster, cluster snapshot generation operations are continuously performed to form a snapshot sequence containing multiple snapshot versions, and retention and deletion operations are performed in the object storage; When the distributed database cluster fails, a recovery process is executed based on the cluster snapshot in the object storage; After the distributed database cluster is restored, the node address information of the distributed database cluster is updated according to the node configuration in the new deployment environment. The step of performing the recovery process based on the cluster snapshot in the object storage specifically includes: Load the latest retained metadata snapshot from the current snapshot sequence from the object storage, and restore the metadata state of the distributed database cluster based on the latest retained metadata snapshot to obtain the restored metadata information; Based on the latest transaction index and status information recorded in the latest retained metadata snapshot, the incremental log is selectively replayed; After restoring the metadata status of the distributed database cluster, the corresponding version of the data file in the object storage is accessed based on the restored metadata information to restore the normal operation capability of the distributed database cluster. The step of updating the node address information of the distributed database cluster according to the node configuration in the new deployment environment specifically includes: Based on the instance identifiers and corresponding network connection parameters of each node in the newly deployed environment, a node address mapping table is generated for the current environment, wherein each node includes compute nodes and metadata management nodes. Based on the node address mapping table, the original node identifier in the latest retained metadata snapshot is replaced to obtain the updated node address information; The updated node address information is synchronized to each metadata management node and data computing node in the newly deployed environment.

2. The cluster snapshot-based distributed database system disaster recovery method of claim 1, wherein, The generation of cluster snapshots based on the storage-compute separation architecture specifically includes: In the metadata management node deployed in the distributed database cluster, a full metadata snapshot is generated based on the distributed consensus protocol. The full metadata snapshot includes data sharding information, data version identifier, scheduling control information, and transaction index and status information for log replay. The full metadata snapshot is used as the metadata part of the cluster snapshot. In the data computing nodes of the distributed database cluster, the multi-version data files generated during the operation of the distributed database cluster are written to the object storage for storage, forming the data portion of the cluster snapshot; Based on the full metadata snapshot, the corresponding target data version set is recorded, and the target data version set is used as the range boundary of the data snapshot in the cluster snapshot. 3.The cluster snapshot-based disaster recovery method for distributed database system according to claim 2, wherein, The generation of a full metadata snapshot based on the distributed consensus protocol specifically includes: In the metadata management node, a full metadata snapshot is generated by executing a replica state synchronization mechanism based on the distributed consensus protocol, wherein the distributed consensus protocol includes the Raft consensus algorithm or the Paxos consensus algorithm. When the preset triggering conditions for snapshot generation are met, a snapshot file of the current consistency state is generated. The snapshot file serves as the full metadata snapshot. The preset triggering conditions include the snapshot period reaching a preset time interval.

4. The cluster snapshot-based distributed database system disaster recovery method of claim 3, wherein, The continuous execution of cluster snapshot generation operations, forming a snapshot sequence containing multiple snapshot versions, and the performance of retention and deletion operations in the object storage, specifically includes: A cluster snapshot generation task is triggered according to the preset time interval; After completing a new round of cluster snapshot generation, the number of existing snapshot copies in the object storage is determined based on a preset retention strategy. When the number of snapshot replicas does not exceed the preset retention limit, all existing cluster snapshots are retained to form a snapshot sequence. When the number of snapshot replicas exceeds the preset retention limit, the earliest generated cluster snapshot is deleted in the order of snapshot creation time.

5. The disaster recovery method for a distributed database system based on cluster snapshots according to claim 1, characterized in that, The step of synchronizing the updated node address information to each metadata management node and data computing node in the newly deployed environment specifically includes: During the synchronization of the updated node address information, if it is detected that any target node has failed to receive the updated node address information, the synchronization operation will be re-initiated based on a preset retry mechanism, which includes a maximum number of retry attempts and an interval time strategy. If node address synchronization is not completed after reaching the maximum number of retries, the status information of the corresponding failed node is recorded, and a disaster recovery alarm is triggered.

6. The disaster recovery method for a distributed database system based on cluster snapshots according to claim 1, characterized in that, The disaster recovery method for distributed database systems based on cluster snapshots also includes: The cluster snapshots in the object storage are periodically synchronized to the target region object storage service deployed in a different location, and the snapshot storage structure and naming path in the target region object storage are kept consistent with those in the object storage. The synchronization operation of the cluster snapshots includes full replication mode and incremental replication mode.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the disaster recovery method for a distributed database system based on cluster snapshots as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the disaster recovery method for a distributed database system based on cluster snapshots as described in any one of claims 1 to 6.

Images