Database data backup method, device, equipment and storage medium
By dynamically selecting the best-performing standby node for backup in a distributed database system, creating data consistency points and recording transaction logs, the data discrepancy problem caused by inconsistent backup speeds is resolved, achieving efficient and reliable data backup and recovery.
Patent Information
- Application Number
- CN202510875400.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-09-16
- Estimated Expiration
- 2045-06-27
AI Technical Summary
In a distributed database system, inconsistent backup speeds across shard clusters lead to data discrepancies, impacting the accuracy and reliability of backup data. Relying solely on the master node for backup reduces overall backup efficiency.
By dynamically selecting the best performing standby node in each shard cluster for backup, a data consistency point is created, marking the moment when the node data is consistent, and the consistency point is synchronously distributed to each shard cluster, and transaction logs are recorded for persistent storage.
Significantly improve backup efficiency, ensure data integrity and consistency, provide accurate and reliable data recovery protection, optimize backup performance and improve resource utilization.
Smart Images

Figure CN120371612B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of cloud computing, and more specifically, to a database data backup method, apparatus, device, and storage medium. Background Art
[0002] In a distributed database system, a service cluster consists of multiple sharded clusters, each of which contains a master node and a backup node. To ensure data security, backup operations are typically performed by the master node. In the event of a failure, the system can quickly restore data based on the master node's backup, ensuring stable system operation and maintaining data integrity.
[0003] During the implementation of the present invention, we discovered at least the following issues with related technologies: During data backup, inconsistent backup speeds across sharded clusters can lead to data discrepancies between them. Furthermore, relying solely on the master node for backup not only reduces overall backup efficiency but also compromises the accuracy and reliability of the backed-up data. Summary of the Invention
[0004] In view of this, the present invention provides a database data backup method, apparatus, device, medium and program product.
[0005] One aspect of the present invention provides a database data backup method, comprising: in response to a received data backup request, dynamically selecting a target node for each of the above shard clusters based on the performance indicators of the standby nodes in each shard cluster in the server cluster; creating a data consistency point when it is determined that the target node in each of the above shard clusters has completed the basic data backup of the master node, and the above data consistency point is used to mark the moment when the data of each node in the above server cluster reaches a consistent state; synchronously distributing the above data consistency point to each of the above shard clusters so that each of the above shard clusters records the transaction log from the backup end time of the above target node to the data consistency point; and persistently storing the above basic data and the above transaction log.
[0006] According to an embodiment of the present invention, the above-mentioned performance indicators include sub-indicator data of multiple dimensions; the above-mentioned target nodes of each shard cluster in the server cluster are dynamically selected based on the performance indicators of the standby nodes in each shard cluster, including: for each of the above-mentioned shard clusters, based on the sub-indicator data of each of the above-mentioned standby nodes, a multi-dimensional sorting algorithm is used to generate sorting results corresponding to each dimension, and the above-mentioned sub-indicator data include read and write speed, processor utilization and data playback speed of the standby node; according to preset weights, the sorting results of each of the above-mentioned standby nodes in each dimension are weighted and fused to generate a comprehensive performance score; according to the above-mentioned comprehensive performance score, a target node is selected from the multiple standby nodes of the above-mentioned shard cluster.
[0007] According to an embodiment of the present invention, the above method also includes: determining the node status of each of the above-mentioned standby nodes based on the performance indicators of the standby nodes in each of the above-mentioned shard clusters; for each of the above-mentioned shard clusters, when it is determined that the node status of each of the above-mentioned standby nodes is in an unavailable state, determining the master node in the above-mentioned shard cluster as the above-mentioned target node.
[0008] According to an embodiment of the present invention, the above-mentioned creation of the data consistency point includes: randomly selecting a coordination node from multiple nodes in the running state according to the node status of each node in the above-mentioned server cluster, and the above-mentioned coordination node is a master node or a backup node; sending a creation request of the above-mentioned data consistency point to the above-mentioned coordination node, so that the above-mentioned coordination node generates a globally unique identifier including a timestamp according to the current system time after receiving the above-mentioned creation request.
[0009] According to an embodiment of the present invention, the above method also includes: when it is determined that the above coordination node has received the above creation request, sending a blocking signal to the master node in each of the above shard clusters, and the above blocking signal is used to block the submission of subsequent transactions; when each of the above shard clusters receives the above blocking signal, stopping receiving the submission request of the above subsequent transactions; and completing the pending transactions that have started to be executed in the above shard cluster; when it is determined that the pending transactions of multiple of the above shard clusters have been executed, generating the above data consistency point through the above coordination node.
[0010] According to an embodiment of the present invention, the above method also includes: storing the log information generated by the master node in each of the above sharded clusters when processing transactions to a log archive server, and the above log information includes detailed information on the transaction operations performed by the master node at each moment.
[0011] According to an embodiment of the present invention, the above method also includes: in response to a received data recovery request, verifying the recovery time in the above data recovery request to obtain a verification result; when it is determined that the above verification result indicates that the above recovery time is less than or equal to the above backup end time, restoring the above basic data; when it is determined that the above verification result indicates that the above recovery time is greater than the above backup end time, for each of the above shard clusters, searching the archive log of the above master node for a data consistency point that is closest to the above recovery time and shared by multiple above shard clusters; when it is determined that the found data consistency point is consistent with the above recovery time, restoring the above basic data; and replaying the above transaction log to obtain restored data; when it is determined that the found data consistency point is inconsistent with the above recovery time, restoring the above basic data; and retrieving log information between the above recovery time and the found data consistency point from the above log archive server; replaying the above transaction log and the retrieved log information to obtain restored data.
[0012] Another aspect of the present invention provides a database data backup device, including: a node selection module, which is used to respond to the received data backup request and dynamically select the target node of each of the above shard clusters according to the performance indicators of the standby nodes in each shard cluster in the server cluster; a data point creation module, which is used to create a data consistency point when it is determined that the target node in each of the above shard clusters has completed the basic data backup of the master node, and the above data consistency point is used to mark the moment when the data of each node in the above server cluster reaches a consistent state; a data point distribution module, which is used to synchronously distribute the above data consistency point to each of the above shard clusters, so that each of the above shard clusters records the transaction log from the backup end time of the above target node to the data consistency point; a data storage module, which is used to persistently store the above basic data and the above transaction log.
[0013] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.
[0014] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, wherein the instructions are used to implement the above method when executed.
[0015] Another aspect of the present invention provides a computer program product, which includes computer executable instructions. When the instructions are executed, they are used to implement the method described above.
[0016] According to an embodiment of the present invention, by dynamically selecting the best-performing backup node in each shard cluster for backup, the target node is accurately located, thereby significantly improving backup efficiency and minimizing the impact on the business. After completing the basic data backup, a data consistency point is created to mark the data consistency status of each node in the shard cluster, ensuring the integrity and consistency of the backup data. Subsequently, the data consistency point is synchronously distributed to each shard cluster, and the transaction log from the end of the backup to the data consistency point is recorded to provide accurate and reliable protection for data recovery. Finally, the basic data and transaction logs are persistently stored to optimize backup performance while ensuring data security, thereby achieving efficient use of cluster resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The above and other objects, features and advantages of the present invention will become more apparent from the following description of the embodiments of the present invention with reference to the accompanying drawings.
[0018] Figure 1An exemplary system architecture to which the database data backup method and apparatus of the present invention can be applied is shown.
[0019] Figure 2 A flowchart of a database data backup method according to an embodiment of the present invention is shown.
[0020] Figure 3 A schematic diagram of basic data backup in a database data backup method according to an embodiment of the present invention is shown.
[0021] Figure 4 A schematic diagram showing log information storage in a database data backup method according to an embodiment of the present invention is shown.
[0022] Figure 5 A flowchart of data recovery according to an embodiment of the present invention is shown.
[0023] Figure 6 A block diagram of a database data backup device according to an embodiment of the present invention is shown.
[0024] Figure 7 A block diagram of an electronic device suitable for implementing a database data backup method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0025] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessarily confusing the concept of the present invention.
[0026] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.
[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0028] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0029] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of all data involved (including, but not limited to, user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken to prevent unauthorized access to user personal information data and maintain the security of user personal information and network security.
[0030] In the embodiment of the present invention, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.
[0031] Embodiments of the present invention provide a database data backup method, apparatus, device, and storage medium. The method can be applied to the field of cloud computing. The method includes: in response to a received data backup request, dynamically selecting a target node for each shard cluster based on the performance indicators of the standby nodes in each shard cluster in the server cluster; creating a data consistency point when it is determined that the target node in each shard cluster has completed the basic data backup of the master node. The data consistency point is used to mark the moment when the data of each node in the server cluster reaches a consistent state; synchronously distributing the data consistency point to each shard cluster so that each shard cluster records the transaction log from the backup end time of the target node to the data consistency point; and persistently storing the basic data and transaction log.
[0032] Figure 1 The following shows an exemplary system architecture to which the database data backup method and apparatus of the present invention can be applied. Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present invention may be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention cannot be used in other devices, systems, environments or scenarios.
[0033] like Figure 1As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.
[0034] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).
[0035] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0036] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.
[0037] It should be noted that the database data backup method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the database data backup device provided in the embodiment of the present invention can generally be set in the server 105. The database data backup method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the database data backup device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the database data backup method provided in the embodiment of the present invention can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Correspondingly, the database data backup device provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.
[0038] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.
[0039] Figure 2 A flowchart of a database data backup method according to an embodiment of the present invention is shown.
[0040] like Figure 2 As shown, the method includes operations S210 to S240.
[0041] In operation S210 , in response to the received data backup request, a target node of each shard cluster is dynamically selected based on the performance indicators of the backup nodes in each shard cluster in the server cluster.
[0042] In operation S220, when it is determined that the target node in each shard cluster completes the basic data backup of the master node, a data consistency point is created. The data consistency point is used to mark the moment when the data of each node in the server cluster reaches a consistent state.
[0043] In operation S230 , the data consistency point is synchronously distributed to each shard cluster so that each shard cluster records a transaction log from the backup end time of the target node to the data consistency point.
[0044] In operation S240 , the basic data and the transaction log are persistently stored.
[0045] According to embodiments of the present invention, in a distributed database, a service cluster typically consists of multiple sharded clusters. To ensure data reliability, each sharded cluster contains multiple replicas, divided into primary and backup nodes. The primary node is responsible for executing write operations, while the backup node serves as a replica of the primary node. If the primary node experiences an exception, the backup node takes over its responsibilities and continues to provide services. Furthermore, to further ensure data security, a physical backup of the entire database is required, typically performed on the primary node. However, to reduce pressure on the primary node, consider using a backup node for backup to improve resource utilization.
[0046] When backup targets are backup nodes, the backup process must ensure data consistency across all sharded clusters, as data synchronization speeds vary between backup nodes within each sharded cluster. During backups using backup nodes, a data backup request is first received from the user. Next, performance metrics are obtained for all backup nodes in each sharded cluster within the server cluster. These metrics may include read / write speeds, processor utilization, and data playback speed. Based on these performance metrics, a target node is selected for each sharded cluster—the node with the best performance among the multiple backup nodes. For example, the target node might be one with faster read / write speeds or one with lower processor utilization to ensure efficient and stable backups.
[0047] Based on the target nodes selected by each sharded cluster, a parallel basic backup is triggered for each sharded cluster to back up the basic data generated by the master node during transaction processing. Specifically, a backup command is sent to the backup module in each target node. Upon receiving the backup request, the backup module initiates the backup process, backing up database data and log information. Upon completion, the user is notified of the backup completion. This parallel backup mechanism allows each sharded cluster to independently complete its backup task, reducing backup wait time and significantly improving backup speed.
[0048] After confirming that the target nodes in each sharded cluster have completed backing up the primary node's basic data, a data consistency point is created. A data consistency point marks the moment when data across nodes in the server cluster reaches a consistent state, typically represented by a globally unique identifier with a timestamp. Creating a data consistency point ensures that data across nodes in each sharded cluster is fully synchronized at a specific point in time, providing a reliable benchmark for subsequent data recovery or consistency verification.
[0049] After each shard cluster completes all pending transactions, a data consistency point is created and synchronously distributed to all shard clusters. Since all transactions have been processed, this ensures that the transaction states of all nodes are consistent. The data consistency point is then recorded in the archive logs of all nodes in each shard cluster. After the base data backup is complete, the base data and the transaction logs recorded by each shard cluster from the backup completion time to the data consistency point are persistently stored in the storage unit, completing the backup process.
[0050] like Figure 3 As shown, the architecture for implementing data backup includes three key modules: a backup machine selection module 310, a backup execution module 320, and a consistency point module 330. These modules work together to ensure the reliability and consistency of data in a distributed environment.
[0051] The backup node selection module 310 is responsible for selecting appropriate nodes from shard cluster 1 and shard cluster 2 for backup operations. In a distributed database system, data is divided into multiple shard clusters, each of which consists of a master node 311 and multiple backup nodes 312. The backup node selection module 310 selects the most suitable node from these nodes to perform the backup operation based on the current system status and backup requirements.
[0052] The backup execution module 320 is responsible for the actual backup operation. It works in conjunction with the node selected by the backup machine selection module 310 to back up the data on the selected node. Figure 3 As can be seen in the figure, backup execution module 320 is connected to shard cluster 1 and shard cluster 2. Each shard cluster includes a primary node and a backup node. Backup execution module 320 performs backup operations on these primary and backup nodes to ensure data integrity and consistency. Simultaneously, backup module 321 is responsible for securely transferring backup data to a storage location to prevent data loss.
[0053] The consistency point module 330 ensures that the data in shard cluster 1 and shard cluster 2 are in a consistent state during the backup process. In a distributed system, where data is distributed across multiple nodes, maintaining data consistency is a challenge. The consistency point module 330 addresses this problem by creating data consistency points, which mark specific moments when the database states are consistent. During the backup process, the consistency point module 330 ensures that the data in all shard clusters is fully synchronized at a specific point in time, providing a reliable baseline for data recovery.
[0054] Through the close collaboration of three key modules—the backup machine selection module 310, the backup execution module 320, and the consistency point module 330—the distributed database system ensures data security and consistency. This architecture not only improves data backup reliability but also significantly enhances the system's resilience to failures, making data management more robust and efficient.
[0055] According to an embodiment of the present invention, by dynamically selecting the best-performing backup node in each shard cluster for backup, the target node is accurately located, thereby significantly improving backup efficiency and minimizing the impact on the business. After completing the basic data backup, a data consistency point is created to mark the data consistency status of each node in the shard cluster, ensuring the integrity and consistency of the backup data. Subsequently, the data consistency point is synchronously distributed to each shard cluster, and the transaction log from the end of the backup to the data consistency point is recorded to provide accurate and reliable protection for data recovery. Finally, the basic data and transaction logs are persistently stored to optimize backup performance while ensuring data security, thereby achieving efficient use of cluster resources.
[0056] According to an embodiment of the present invention, the performance indicator includes sub-indicator data of multiple dimensions; based on the performance indicators of the standby nodes in each shard cluster in the server cluster, the target node of each shard cluster is dynamically selected, including: for each shard cluster, based on the sub-indicator data of each standby node, a multi-dimensional sorting algorithm is used to generate sorting results corresponding to each dimension, and the sub-indicator data includes read and write speed, processor utilization and data playback speed of the standby node; according to preset weights, the sorting results of each standby node in each dimension are weighted and fused to generate a comprehensive performance score; according to the comprehensive performance score, the target node is selected from the multiple standby nodes in the shard cluster.
[0057] According to an embodiment of the present invention, for each sharded cluster, the sub-indicator data of each standby node is first obtained, including the read and write speed, processor utilization, and data playback speed of the standby node. These indicators are key factors in evaluating the performance of the standby node. For example, suppose there are three standby nodes in a sharded cluster, namely standby node A, standby node B, and standby node C. The data obtained through the monitoring tool are as follows: the read and write speed of standby node A is 100MB / s, the processor utilization is 30%, and the data playback speed is 50MB / s; the read and write speed of standby node B is 80MB / s, the processor utilization is 40%, and the data playback speed is 60MB / s; the read and write speed of standby node C is 90MB / s, the processor utilization is 25%, and the data playback speed is 45MB / s.
[0058] Based on these sub-metric data, a multi-dimensional sorting algorithm is used to generate ranking results corresponding to each dimension. Specifically, read and write speeds are sorted first, with higher read and write speeds ranking higher. In the above example, the ranking results are: backup node A (100MB / s) > backup node C (90MB / s) > backup node B (80MB / s). Next, processor utilization is sorted. Lower processor utilization indicates a less busy node and is ranked higher. Therefore, the ranking results are: backup node C (25%) > backup node A (30%) > backup node B (40%). Finally, data playback speed is sorted. Higher data playback speeds are ranked higher, resulting in the following ranking results: backup node B (60MB / s) > backup node A (50MB / s) > backup node C (45MB / s).
[0059] Based on preset weights, the ranking results for each standby node in each dimension are weighted and combined to generate a comprehensive performance score. Assume that the preset weights are: read / write speed = 0.4, processor utilization = 0.3, and data playback speed = 0.3. For standby node A, its comprehensive performance score is: (1 × 0.4) + (2 × 0.3) + (2 × 0.3) = 1.6. Similarly, the comprehensive performance scores for standby nodes B and C are: (3 × 0.4) + (3 × 0.3) + (1 × 0.3) = 2.4, and (2 × 0.4) + (1 × 0.3) + (3 × 0.3) = 2.0, respectively.
[0060] Finally, a target node is selected from the multiple standby nodes in the sharded cluster based on the overall performance score. In the example above, standby node A has the lowest overall performance score (1.6) and is therefore selected as the target node. This ensures that the selected target node performs optimally across multiple key performance indicators, providing optimal performance support for subsequent backup and restore operations.
[0061] According to an embodiment of the present invention, the database data backup method also includes: determining the node status of each standby node based on the performance indicators of the standby nodes in each shard cluster; for each shard cluster, when it is determined that the node status of each standby node is in an unavailable state, determining the master node in the shard cluster as the target node.
[0062] According to an embodiment of the present invention, before selecting a target node, it is first necessary to determine the node status of each standby node based on the performance indicators of the standby nodes in each sharded cluster. Specifically, the performance indicators such as the read and write speed, processor utilization and data playback speed of each standby node are detected in real time through a monitoring tool or a health check mechanism. If the performance indicator of a standby node cannot be obtained, or its response times out, it can be determined that the standby node is in an unavailable state. For example, suppose there are three standby nodes in a sharded cluster, namely standby node A, standby node B and standby node C. During the detection process, it was found that the read and write speed of standby node A was 100MB / s, the processor utilization was 30%, and the data playback speed was 50MB / s; the read and write speed of standby node B was 80MB / s, the processor utilization was 40%, and the data playback speed was 60MB / s; and standby node C was unable to respond to the detection request, so it was determined that standby node C was in an unavailable state.
[0063] For each sharded cluster, after determining the node status of all standby nodes, if it is found that all standby nodes are in an unavailable state, that is, they are inaccessible or their performance indicators cannot be obtained, the master node in the sharded cluster is determined as the target node. The master node generally has higher performance and stability, and can serve as a reliable choice for backup operations when all standby nodes are unavailable. For example, in the sharded cluster described above, if standby nodes A and B are also inaccessible due to network failure or other reasons, the system will automatically determine the master node as the target node to ensure that the backup operation can proceed smoothly. In this way, if the standby node is unavailable, it can quickly switch to the master node, avoiding the interruption of the backup operation due to node unavailability, thereby improving the availability and reliability of the entire distributed database.
[0064] According to an embodiment of the present invention, creating a data consistency point includes: randomly selecting a coordination node from multiple nodes in a running state according to the node status of each node in the server cluster, the coordination node being a master node or a backup node; sending a data consistency point creation request to the coordination node, so that after receiving the creation request, the coordination node generates a globally unique identifier including a timestamp according to the current system time.
[0065] According to an embodiment of the present invention, in a distributed database environment, the creation of a data consistency point is a key step to ensure data synchronization and backup. First, it is necessary to randomly select a coordination node from multiple nodes in a running state based on the node status of each node in the server cluster. The coordination node can be a master node or a standby node. For example, in a distributed database system containing multiple shard clusters, each shard cluster has several master nodes and standby nodes. Assume that there are master nodes M1, standby nodes S1, standby nodes S2 and standby nodes S3 in the current cluster, where S3 is in an unavailable state due to a fault, while M1, S1 and S2 are all in a running state. At this time, the system will randomly select a node from M1, S1 and S2 as the coordination node. Assume that the result of the random selection is standby node S1.
[0066] After determining the coordinating node, a request to create a data consistency point is sent to the coordinating node. After receiving the creation request, the coordinating node generates a globally unique identifier (BarrierID) containing a timestamp based on the current system time. This globally unique identifier is used to identify the consistency point of the current transaction and includes the current time information to ensure its uniqueness and traceability. For example, assuming the current system time is 14:30:00 on May 15, 2025, after coordinating node S1 receives the creation request, the generated Barrier ID may be Barrier-20250515143000-12345678, where 20250515143000 represents the generation time and 12345678 is a randomly generated unique sequence number.
[0067] The creation of a data consistency point not only ensures the synchronization of data across nodes within a sharded cluster at a specific point in time, but also provides a clear benchmark for subsequent data recovery and consistency verification through a globally unique identifier. This mechanism effectively ensures data integrity and consistency in complex distributed environments. Furthermore, by randomly selecting coordination nodes, it avoids over-reliance on a single node, improving system reliability and availability.
[0068] According to an embodiment of the present invention, the database data backup method also includes: when it is determined that the coordination node has received a creation request, sending a blocking signal to the master node in each shard cluster, the blocking signal is used to block the submission of subsequent transactions; when each shard cluster receives the blocking signal, stopping receiving submission requests for subsequent transactions; and completing the pending transactions that have started to be executed in the shard cluster; when it is determined that the pending transactions of multiple shard clusters have been executed and completed, generating a data consistency point through the coordination node.
[0069] According to an embodiment of the present invention, after determining that the coordinating node has received a request to create a data consistency point, it will send a blocking signal to the master node in each shard cluster. The purpose of this blocking signal is to temporarily stop the submission of subsequent transactions to ensure that the transaction status within the shard cluster remains stable during the process of creating the data consistency point. For example, suppose that after receiving the creation request, the coordinating node S1 sends a blocking signal to the master node M1 of shard cluster 1, the master node M2 of shard cluster 2, and the master node M3 of shard cluster 3. After receiving the blocking signal, these master nodes will immediately stop receiving new transaction submission requests.
[0070] After receiving the blocking signal, each shard cluster's master node will stop accepting subsequent transaction commit requests, but will continue to complete pending transactions that have already started. This process ensures that all ongoing transactions can be successfully completed during the blocking state, avoiding data inconsistencies caused by transaction pauses. For example, in shard cluster 1, master node M1 may be processing three pending transactions. After receiving the blocking signal, M1 will stop accepting new transaction commit requests but will continue to complete the three already started transactions. Similarly, the master nodes of shard clusters 2 and 3 will complete pending transactions that have already started in their respective shard clusters.
[0071] After confirming that all pending transactions across multiple sharded clusters have completed, the coordinating node generates a data consistency point. At this point, since all pending transactions have completed, the transaction states within the sharded clusters are consistent. The coordinating node generates a globally unique identifier (GUI) containing a timestamp based on the current system time to identify the data consistency point. This GUI not only identifies the data consistency point but also records the creation time, providing a reliable benchmark for subsequent data recovery and consistency verification.
[0072] Through this series of steps, the data consistency point creation process ensures that the transaction status of all nodes in a distributed database system is completely consistent at a specific point in time. This mechanism effectively avoids data inconsistencies caused by uncertainty in transaction submission, improving the reliability and data integrity of the distributed database system.
[0073] According to an embodiment of the present invention, the database data backup method also includes: storing the log information generated by the master node in each shard cluster when processing transactions to a log archive server, where the log information includes detailed information on the transaction operations performed by the master node at each moment.
[0074] According to an embodiment of the present invention, in a distributed database system, the backup module of a standby node receives a user's data backup request. If the user requests a point-in-time recovery (PITR) backup, the backup module triggers the distributed database's log archiving operation. The core of this operation is to continuously archive the log information generated by the master node during transaction processing to a log archiving server to support subsequent point-in-time recovery requirements.
[0075] Specifically, log archiving involves the master nodes in each sharded cluster. These master nodes generate extensive log information when processing transactions. This information records the details of each transaction performed by the master node at each moment. For example, log information may include the transaction start and end time, the data tables involved, the operation type (such as insert, update, or delete), and the transaction results. This log information is the foundation of point-in-time recovery backups, as it allows the system to be restored to a specific point in time when needed.
[0076] To ensure the integrity and availability of log information, the master node continuously sends this log information to the log archive server. The log archive server is a server dedicated to storing and managing log information, and it features high availability and large storage capacity. Once stored on the log archive server, log information can be used for subsequent backup operations, data recovery, and consistency verification. For example, if a user needs to restore data to a specific point in time, the system can reconstruct the data state at that point in time using the log information on the log archive server, thus enabling point-in-time recovery.
[0077] In this way, distributed database systems can effectively support point-in-time recovery backups, ensuring that in the event of data loss or operational errors, users can quickly restore data to the state at any specified point in time. This mechanism not only improves system reliability but also enhances the flexibility and security of data management.
[0078] like Figure 4 As shown, in the data backup architecture, in addition to the standby node selection module, backup execution module, and consistency point module, a log archive server 410 is introduced to support point-in-time recovery (PITR) operations. Log archive server 410 stores log information archived from the primary node, which is crucial for restoring data to a specific point in time.
[0079] After completing the data backup, the backup execution module transfers the backup data to the log archive server 410. This backup data includes all data states prior to the data consistency point created by the consistency point module. When a user requests a point-in-time restore, this backup data and the log information in the log archive server 410 are used to restore the data to the user-specified point in time.
[0080] To achieve point-in-time recovery, the user's requested recovery time is first determined, and then the data consistency point closest to that time point is searched. Once found, log records prior to that time point are retrieved from the log archive server 410 and replayed based on the backup data until the user's requested recovery time is reached. This process ensures that data can be accurately restored to the specified time point, meeting the user's requirements for data recovery accuracy.
[0081] This data backup architecture enables distributed database systems to not only perform regular data backups but also support precise point-in-time recovery operations. This significantly enhances the system's data recovery capabilities, allowing users to quickly restore to any historical state in the event of data loss or corruption, thereby ensuring business continuity and data integrity.
[0082] Figure 5 A flowchart of data recovery according to an embodiment of the present invention is shown.
[0083] like Figure 5 As shown, the process includes operations S501 to S508.
[0084] In operation S501 , in response to a received data recovery request, the recovery time in the data recovery request is verified to obtain a verification result.
[0085] In operation S502 , it is determined whether the restoration time is greater than the backup end time.
[0086] In operation S503 , basic data is restored.
[0087] In operation S504 , a data consistency point that is closest to the recovery time and is shared by multiple shard clusters is searched from the archive logs of the master node in each shard cluster.
[0088] In operation S505 , it is determined whether the found data consistency point is consistent with the recovery time.
[0089] In operation S506 , the basic data is restored and the transaction log is replayed.
[0090] In operation S507 , log information between the recovery time and the found data consistency point is retrieved from the log archive server.
[0091] In operation S508 , the basic data is restored, and the transaction log and the retrieved log information are replayed.
[0092] According to an embodiment of the present invention, when a data recovery request is received, the recovery time in the request is first verified. If the recovery time is less than or equal to the backup end time, the basic data is directly restored. For example, assuming the backup end time is 14:00:00 on May 15, 2025, and the user requests to restore to the data state at 13:00:00 on May 15, 2025. In this case, the recovery time is earlier than the backup end time, so the basic data can be directly read from the backup storage unit and restored to the corresponding nodes of each shard cluster. This recovery method is suitable for users who need to restore to the state before the backup time point. The operation is simple and efficient.
[0093] After restoring the base data, the data consistency of each shard cluster is verified to ensure that all nodes are in the same state. Because the recovery time is within the valid range of the backup data, no additional log replay is required, resulting in a rapid recovery process. This method is particularly suitable for users who need to quickly restore to a state before the backup time point and do not require high recovery time accuracy.
[0094] According to an embodiment of the present invention, if the recovery time is greater than the backup end time, a point-in-time recovery (PITR) is required. First, for each sharded cluster, the archived logs of the master node are searched for a data consistency point closest to the recovery time that is shared by multiple sharded clusters. For example, suppose a user requests a recovery to the data state at 15:00:00 on May 15, 2025, and the backup end time is 14:00:00 on May 15, 2025. In this case, the archived logs are searched for the data consistency point closest to 15:00:00.
[0095] During the search process, you may encounter inconsistent data consistency points across different shard clusters. For example, the most recent data consistency point for shard cluster 1 is Barrier-20250515143000, while the most recent data consistency point for shard cluster 2 is Barrier-20250515144500. This inconsistency may be due to different transaction processing speeds across the shard clusters. In this case, you need to continue searching for the previous data consistency point until you find a unified data consistency point that exists across all shard clusters. Assuming the final unified data consistency point is Barrier-20250515141500, this point will be used as the recovery target for the shard clusters.
[0096] If the found data consistency point is consistent with the recovery time, the base data is restored first, and then the transaction logs are replayed until the recovery time is reached. If the found data consistency point is inconsistent with the recovery time, log information between the recovery time and the found data consistency point is retrieved from the log archive server. For example, if the recovery time is 15:00:00 and the unified data consistency point is Barrier-20250515141500, log information between 14:15:00 and 15:00:00 is retrieved. The transaction logs and the retrieved log information are replayed until the recovery time is reached, completing the recovery operation.
[0097] In this way, each sharded cluster first restores the backup data, then replays the archived logs until they reach a unified data consistency point. Because all sharded clusters replay to the same data consistency point, data consistency is guaranteed after recovery. This mechanism not only improves the flexibility of the recovery process, but also enhances the reliability and accuracy of data recovery.
[0098] Figure 6 A block diagram of a database data backup device according to an embodiment of the present invention is shown.
[0099] like Figure 6 As shown, the database data backup device 600 includes a node selection module 610 , a data point creation module 620 , a data point distribution module 630 , and a data storage module 640 .
[0100] The node selection module 610 is used to respond to the received data backup request and dynamically select the target node of each shard cluster according to the performance indicators of the backup nodes in each shard cluster in the server cluster.
[0101] The data point creation module 620 is used to create a data consistency point when it is determined that the target node in each shard cluster has completed the basic data backup of the master node. The data consistency point is used to mark the moment when the data of each node in the server cluster reaches a consistent state.
[0102] The data point distribution module 630 is used to synchronously distribute the data consistency point to each shard cluster so that each shard cluster records the transaction log from the backup end time of the target node to the data consistency point.
[0103] The data storage module 640 is used to persistently store basic data and transaction logs.
[0104] According to an embodiment of the present invention, the node selection module 610 includes a ranking generation submodule, a score generation submodule, and a node selection submodule.
[0105] The sorting generation submodule is used to generate sorting results corresponding to each dimension for each shard cluster based on the sub-indicator data of each backup node through a multi-dimensional sorting algorithm. The sub-indicator data includes read and write speed, processor utilization, and data playback speed of the backup node.
[0106] The score generation submodule is used to perform weighted fusion on the ranking results of each backup node in each dimension according to preset weights to generate a comprehensive performance score.
[0107] The node selection submodule is used to select the target node from multiple backup nodes in the shard cluster based on the comprehensive performance score.
[0108] According to an embodiment of the present invention, the database data backup device 600 further includes a state determination module and a target determination module.
[0109] The status determination module is used to determine the node status of each standby node based on the performance indicators of the standby nodes in each shard cluster.
[0110] The target determination module is used to determine the master node in each shard cluster as the target node when it is determined that the node status of each backup node is unavailable.
[0111] According to an embodiment of the present invention, the data point creation module 620 includes a coordination selection submodule and a request sending submodule.
[0112] The coordination selection submodule is used to randomly select a coordination node from multiple nodes in the running state according to the node status of each node in the server cluster. The coordination node is the main node or the backup node.
[0113] The request sending submodule is used to send a request for creating a data consistency point to the coordination node, so that the coordination node generates a globally unique identifier including a timestamp according to the current system time after receiving the creation request.
[0114] According to an embodiment of the present invention, the database data backup device 600 further includes a signal sending module, a transaction processing module, and a data point generating module.
[0115] The signal sending module is used to send a blocking signal to the master node in each shard cluster when it is determined that the coordinating node has received the creation request. The blocking signal is used to block the submission of subsequent transactions.
[0116] The transaction processing module is used to stop accepting subsequent transaction submission requests when each shard cluster receives a blocking signal; and complete the pending transactions that have started to be executed in the shard cluster.
[0117] The data point generation module is used to generate data consistency points through the coordination node when it is determined that all pending transactions of multiple shard clusters have been completed.
[0118] According to an embodiment of the present invention, the database data backup device 600 further includes a log archiving module.
[0119] The log archiving module is used to store the log information generated by the master node in each shard cluster when processing transactions to the log archiving server. The log information includes detailed information about the transaction operations performed by the master node at each moment.
[0120] According to an embodiment of the present invention, the database data backup device 600 includes a time verification module, a data recovery module, a data point search module, a transaction playback module, and an information playback module.
[0121] The time verification module is used to verify the recovery time in the data recovery request in response to the received data recovery request to obtain a verification result.
[0122] The data recovery module is used to recover basic data when it is determined that the recovery time indicated by the verification result is less than or equal to the backup end time.
[0123] The data point search module is used to search for the data consistency point closest to the recovery time and shared by multiple shard clusters from the archive log of the master node for each shard cluster when it is determined that the verification result indicates that the recovery time is greater than the backup end time.
[0124] The transaction replay module is used to restore the basic data when it is determined that the data consistency point found is consistent with the recovery time; and replay the transaction log to obtain the restored data.
[0125] The information playback module is used to restore the basic data when it is determined that the data consistency point found is inconsistent with the recovery time; and to retrieve the log information between the recovery time and the data consistency point found from the log archive server; and to replay the transaction log and the retrieved log information to obtain the recovered data.
[0126] Any number of the modules, submodules, units, and subunits according to embodiments of the present invention, or at least part of the functionality of any number of these units, can be implemented in a single module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be split into multiple modules for implementation. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware using any other reasonable method of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be at least partially implemented as a computer program module that, when executed, can perform the corresponding functionality.
[0127] For example, any number of the node selection module 610, the data point creation module 620, the data point distribution module 630, and the data storage module 640 can be combined into a single module / unit / sub-unit, or any one of these modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functionality of one or more of these modules / units / sub-units can be combined with at least part of the functionality of other modules / units / sub-units and implemented in a single module / unit / sub-unit. According to an embodiment of the present invention, at least one of the node selection module 610, the data point creation module 620, the data point distribution module 630, and the data storage module 640 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware by any other reasonable means of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, at least one of the node selection module 610, the data point creation module 620, the data point distribution module 630, and the data storage module 640 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.
[0128] It should be noted that the database data backup device part in the embodiment of the present invention corresponds to the database data backup method part in the embodiment of the present invention. The description of the database data backup device part specifically refers to the database data backup method part and will not be repeated here.
[0129] Figure 7 A block diagram of an electronic device suitable for implementing a database data backup method according to an embodiment of the present invention is shown. Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.
[0130] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage unit 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.
[0131] The RAM 703 stores various programs and data required for the operation of the electronic device 700. The processor 701, ROM 702, and RAM 703 are connected to each other via a bus 704. The processor 701 executes the programs in the ROM 702 and / or RAM 703 to perform the various operations of the method flow according to the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also execute the programs stored in one or more memories to perform the various operations of the method flow according to the embodiment of the present invention.
[0132] According to an embodiment of the present invention, electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to bus 704. Electronic device 700 may also include one or more of the following components connected to I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 708 including a hard disk; and a communication section 709 including a network interface card such as a LAN card or modem. Communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to I / O interface 705 as needed. Removable media 711, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 710 as needed, so that computer programs read from the removable media can be installed into storage section 708 as needed.
[0133] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.
[0134] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.
[0135] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0136] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .
[0137] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the database data backup method provided by the embodiment of the present invention.
[0138] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present invention are performed. According to the embodiment of the present invention, the above-described systems, devices, modules, units, etc. can be implemented by computer program modules.
[0139] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 709, and / or installed from a removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.
[0140] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, languages such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).
[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.
[0142] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.
Claims
1. A database data backup method, characterized in that: include: In response to the received data backup request, dynamically select the target node of each shard cluster according to the performance indicators of the backup nodes in each shard cluster in the server cluster; When it is determined that the target node in each shard cluster completes the basic data backup of the master node and each shard cluster completes all pending transactions, a data consistency point is created, where the data consistency point is used to mark the moment when the data of each node in the server cluster reaches a consistent state; Synchronously distributing the data consistency point to each of the shard clusters so that each of the shard clusters records a transaction log from the backup end time of the target node to the data consistency point; Persistently storing the basic data and the transaction log; In response to the received data recovery request, determine a recovery time, and find a data consistency point that is closest to the recovery time and shared by multiple shard clusters; Obtain log information of the master node in each of the shard clusters before the recovery time, and replay it based on the backup data until the recovery time is reached. The backup data includes all data states before the data consistency point is created.
2. The method according to claim 1, characterized in that The performance indicator includes sub-indicator data of multiple dimensions; the target node of each shard cluster is dynamically selected based on the performance indicator of the standby node in each shard cluster in the server cluster, including: For each of the sharded clusters, based on the sub-index data of each of the standby nodes, a multi-dimensional sorting algorithm is used to generate sorting results corresponding to each dimension, wherein the sub-index data includes read and write speed, processor utilization, and data playback speed of the standby node; Perform weighted fusion on the ranking results of each backup node in each dimension according to preset weights to generate a comprehensive performance score; A target node is selected from the plurality of standby nodes in the shard cluster according to the comprehensive performance score.
3. The method according to claim 2, characterized in that Also includes: Determine the node status of each of the standby nodes based on the performance indicators of the standby nodes in each of the shard clusters; For each of the shard clusters, when it is determined that the node status of each of the standby nodes is in an unavailable state, the master node in the shard cluster is determined as the target node.
4. The method according to claim 1, wherein The step of creating a data consistency point includes: According to the node status of each node in the server cluster, randomly select a coordinating node from multiple nodes whose node status is in a running state, and the coordinating node is a master node or a backup node; A creation request for the data consistency point is sent to the coordination node, so that the coordination node generates a globally unique identifier including a timestamp according to the current system time after receiving the creation request.
5. The method according to claim 4, characterized in that Also includes: When it is determined that the coordinating node has received the creation request, a blocking signal is sent to the master node in each of the shard clusters, where the blocking signal is used to block the submission of subsequent transactions; When each of the shard clusters receives the blocking signal, it stops accepting submission requests for subsequent transactions; And complete the pending transactions that have started to be executed in the shard cluster; When it is determined that all pending transactions of the plurality of shard clusters have been completed, the data consistency point is generated by the coordination node.
6. The method according to claim 1, wherein Also includes: The log information generated by the master node in each of the shard clusters when processing transactions is stored in the log archive server. The log information includes detailed information about the transaction operations performed by the master node at each moment.
7. The method according to claim 6, characterized in that Also includes: In response to the received data recovery request, verifying the recovery time in the data recovery request to obtain a verification result; If it is determined that the verification result indicates that the restoration time is less than or equal to the backup end time, restoring the basic data; If it is determined that the verification result indicates that the recovery time is greater than the backup end time, searching, for each of the shard clusters, from the archive log of the master node for a data consistency point that is closest to the recovery time and is shared by multiple shard clusters; When it is determined that the found data consistency point is consistent with the recovery time, the basic data is restored; and the transaction log is replayed to obtain the restored data; When it is determined that the found data consistency point is inconsistent with the recovery time, restoring the basic data; The log information between the recovery time and the found data consistency point is retrieved from the log archiving server; the transaction log and the retrieved log information are replayed to obtain the recovery data.
8. A database data backup device, characterized in that: include: A node selection module is used to dynamically select a target node for each shard cluster in response to a received data backup request based on the performance indicators of the backup nodes in each shard cluster in the server cluster; A data point creation module is configured to create a data consistency point upon determining that the target node in each shard cluster has completed backing up the basic data of the master node and each shard cluster has completed all pending transactions. The data consistency point is used to mark the moment when the data of each node in the server cluster reaches a consistent state; A data point distribution module, configured to synchronously distribute the data consistency point to each of the shard clusters, so that each of the shard clusters records a transaction log from the backup end time of the target node to the data consistency point; A data storage module, used for persistently storing the basic data and the transaction log; The device is further configured to, in response to a received data recovery request, determine a recovery time, and search for a data consistency point that is closest to the recovery time and shared by multiple shard clusters; Obtain log information of the master node in each of the shard clusters before the recovery time, and replay it based on the backup data until the recovery time is reached. The backup data includes all data states before the data consistency point is created.
9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that Executable instructions are stored thereon, which, when executed by a processor, enable the processor to implement the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Data processing method and device
CN114090332A
Backup method and device, equipment and storage medium
CN116974825A
Backup method of database read-write separation cluster, storage medium and equipment
CN117290160A