Database data recovery systems, methods, devices, and storage media

Abstract

This disclosure provides a database data recovery system, method, apparatus, and storage medium. The database data recovery system, applied to a distributed database, is characterized by comprising: a replica server, a cluster management module, and a target database. The replica server acquires the logical logs of the master database corresponding to the target database within the distributed database and generates at least one log replica based on the logical logs. The cluster management module identifies a target log replica matching the target database from among the at least one log replica. The target database retrieves the target log replica and performs data recovery operations based on it. This eliminates the need for data transfer between the master and target databases, instead directly performing data recovery based on the logical logs, reducing data transfer time and mitigating the risk of data loss.

Description

Technical Field

[0001] This disclosure relates to the field of database technology, and more specifically, to database data recovery systems, methods, apparatus, and storage media. Background Technology

[0002] Currently, most disaster recovery systems in China's financial system adopt off-site disaster recovery solutions, where the main center and the disaster recovery center are deployed in two separate locations. This way, when a disaster occurs at the main center, the production system can be restored at the off-site disaster recovery center, which is located at a distance, and services can continue to be provided to the outside world.

[0003] Recovery Time Object (RTO) and Recovery Point Object (RPO) are commonly used technical metrics for disaster recovery. Ideally, RPO should be zero, and a shorter RTO is better. Disaster recovery centers are typically located far from the primary data center, making it difficult to achieve a zero RPO with off-site data replication solutions. Therefore, asynchronous replication via database can be used when recovering off-site data. However, asynchronous solutions often cannot completely prevent data loss during off-site recovery. For critical data transaction scenarios, even in a disaster, the loss of any single data point can have immeasurable consequences. Summary of the Invention

[0004] This disclosure provides at least a database data recovery system, method, apparatus, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide a data recovery system for a database, applied to a distributed database, including: a replica server, a cluster management module, and a target database;

[0006] The replica server obtains the logical logs of the primary database corresponding to the target database in the distributed database, and generates at least one log replica based on the logical logs;

[0007] The cluster management module identifies a target log replica that matches the target database from at least one of the log replicas;

[0008] The target database is retrieved, a copy of the target log is obtained, and data recovery operations are performed based on the target log copy.

[0009] In one optional implementation, the cluster management module further includes: a cluster manager and a database monitoring agent;

[0010] The cluster manager determines the GTID query instruction based on the master database;

[0011] The database monitoring agent determines the GTID of each log replica based on the GTID query command, identifies the log replica whose GTID value meets the query conditions, and designates the log replica as the target log replica.

[0012] In an optional implementation, the cluster management module is further configured to: after determining the GTID of each log replica, determine a target GTID value, determine the log data corresponding to the target GTID value, and determine a clearing instruction based on the log data, wherein the clearing instruction is used to instruct the database monitoring agent to clear the corresponding log data in each log replica.

[0013] In one optional implementation, the database monitoring agent is further configured to: when the replica server is determined to be the primary database, instruct other servers in the distributed database to perform data processing operations based on the determined target log replica.

[0014] In an optional implementation, the cluster management module is further configured to: upon receiving a primary database switch instruction, determine the log file sequence number of the log replica, and determine a reset instruction based on the log file sequence number, wherein the reset instruction is used to instruct that the log file sequence number be used as the starting sequence number when resetting the log replica.

[0015] In one optional implementation, the data recovery system further includes: a data protection device;

[0016] When data is lost in the replica server, the data protection device, in response to a log recovery command, performs a recovery operation on the log replica in the replica server.

[0017] In one optional implementation, the target database is further configured to: determine the log location corresponding to the target log replica and create a storage path, pull the target log replica based on the storage path, and perform log replay operations based on the target log replica.

[0018] Secondly, this disclosure also provides a data recovery method for a database, applied to a distributed database, including:

[0019] Obtain the logical log of the master database corresponding to the target database in the distributed database, and generate at least one log replica based on the logical log;

[0020] Among the at least one log copy, a target log copy that matches the target database is identified;

[0021] Pull the target log copy and perform data recovery operations in the target database based on the target log copy.

[0022] Thirdly, embodiments of this disclosure also provide a database data recovery apparatus, comprising:

[0023] The acquisition unit is used to acquire the logical logs of the master database corresponding to the target database in the distributed database, and generate at least one log copy based on the logical logs;

[0024] A determining unit is configured to determine, among the at least one log copy, a target log copy that matches the target database;

[0025] The recovery unit is used to pull the target log copy and perform data recovery operations in the target database based on the target log copy.

[0026] Fourthly, embodiments of this disclosure also provide a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps in the embodiments of the second aspect described above are performed.

[0027] Fifthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps described in the second aspect.

[0028] In this embodiment, the database data recovery system includes a replica server, a cluster management module, and a target database. The replica server can obtain the logical logs of the master database corresponding to the target database in the distributed database and generate at least one log replica based on the logical logs. Next, the cluster management module can determine a target log replica that matches the target database from among the at least one log replica. Then, the target database can pull the target log replica and perform data recovery operations based on it. This eliminates the need for data transfer between the master and target databases; instead, data recovery is performed directly based on the logical logs, reducing data transfer time and the risk of data loss.

[0029] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0031] Figure 1 An architecture diagram of a database data recovery system provided according to an embodiment of this disclosure is shown;

[0032] Figure 2 A schematic diagram illustrating the generation of log copies provided in an embodiment of this disclosure is shown;

[0033] Figure 3 This illustration shows a schematic diagram of cleaning up the log copy corresponding to the target shard according to an embodiment of the present disclosure;

[0034] Figure 4 A flowchart of a database data recovery method provided in an embodiment of this disclosure is shown;

[0035] Figure 5 An architectural diagram of the data protection device provided in an embodiment of this disclosure is shown;

[0036] Figure 6 A schematic diagram of a database data recovery apparatus provided in an embodiment of this disclosure is shown;

[0037] Figure 7 A schematic diagram of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0039] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0040] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0041] Research has found that most domestic financial systems currently adopt off-site disaster recovery solutions, where the main center and the disaster recovery center are deployed in two separate locations. This way, when a disaster occurs at the main center, the production system can be restored at the off-site disaster recovery center, which is located at a distance, and services can continue to be provided to the public.

[0042] Recovery Time Object (RTO) and Recovery Point Object (RPO) are commonly used technical metrics for disaster recovery. Ideally, RPO should be zero, and a shorter RTO is better. Disaster recovery centers are typically located far from the primary data center, making it difficult to achieve a zero RPO with off-site data replication solutions. Therefore, asynchronous replication via database can be used when recovering off-site data. However, asynchronous solutions often cannot completely prevent data loss during off-site recovery. For critical data transaction scenarios, even in a disaster, the loss of any single data point can have immeasurable consequences.

[0043] Based on the above research, this disclosure provides a database data recovery system, method, apparatus, and storage medium. In an embodiment of this disclosure, the database data recovery system includes a replica server, a cluster management module, and a target database. The replica server can be used to obtain the logical logs of the master database corresponding to the target database in a distributed database, and generate at least one log replica based on the logical logs. Next, based on the cluster management module, a target log replica matching the target database can be determined from the at least one log replica. Then, the target database can pull the target log replica and perform data recovery operations based on it. This eliminates the need for data transfer between the master and target databases; instead, data recovery is performed directly based on the logical logs, reducing data transfer time and the risk of data loss.

[0044] To facilitate understanding of this embodiment, a detailed description of the database data recovery system disclosed in this disclosure is provided first. The database data recovery system provided in this disclosure is applied to a distributed database, which typically employs a 3AZ deployment architecture. This 3AZ deployment architecture refers to an architecture mode in a cloud computing environment where data is deployed in three availability zones: a local zone, a city-wide zone, and a remote zone. Here, an availability zone refers to a data center within the same geographical area that is isolated from each other and has independent power supply and network. Data can be exchanged between these data centers in different fault domains for data recovery.

[0045] See Figure 1 The diagram shown is an architecture diagram of a database data recovery system provided in this embodiment of the present disclosure. The system is applied to a distributed database and includes: a replica server 10, a cluster management module 20, and a target database 30.

[0046] Replica Server 10: Obtains the logical logs of the primary database corresponding to the target database in the distributed database, and generates at least one log replica based on the logical logs.

[0047] In the embodiments disclosed herein, distributed databases typically use smaller computer systems, with each computer located in a separate location. Each computer may contain a complete or partial copy of the DBMS (Database Management System) and has its own local database. Many computers located in different locations are interconnected through a network to form a complete, global, logically centralized, and physically distributed large database.

[0048] Specifically, a distributed database system may include at least one master database and multiple slave databases. This distributed data system can be divided into multiple shards based on geographical regions, with each shard including one master database and corresponding slave databases. Here, the master database can store logical logs of the data processing process, while the slave databases can store relay logs, the data in which can be a subset of the logical logs.

[0049] Based on this, logical logs can be retrieved from the primary database via a replica server, and log replicas can be generated based on these logical logs. It should be understood that log replicas are created based on a lightweight principle, meaning that SQL threads are not started and the internal logical logging function is not enabled, thus retaining only the logical logs without saving the actual data.

[0050] When generating log replicas based on logical logs, a functional tag can first be added to the log replica to add it to the replica server. Specifically, the distributed databases mentioned above can first be grouped. As shown above, distributed databases typically adopt a 3AZ deployment architecture, which includes three availability zones: local zone, intra-city zone, and remote zone. Therefore, the distributed databases can be grouped based on this 3AZ deployment architecture. For example, the local zone and intra-city zone can be grouped together, and the remote zone can be grouped together, with each group corresponding to at least one log replica.

[0051] It should be understood that local and intra-city areas are typically located in the same city, while remote areas are in different cities. Therefore, following the grouping method described above allows two groups to serve as off-site data backups for each other. Specifically, after determining the log copy, it can be replicated to obtain multiple log copies, and at least one log copy can be allocated to each group, thereby improving the success rate of data recovery after a disaster.

[0052] like Figure 2 The diagram illustrates the process of generating log replicas. The replica server connects to the primary database and establishes a replication relationship. Then, an I / O thread replicates the logical logs from the primary database to the log replica, creating a relay log. Specifically, the replica server can start a worker thread, the I / O thread, which establishes a client connection with the primary database and starts a binary dump thread on the primary database. This dump thread reads the logical logs from the primary database.

[0053] When data is lost in a database within a distributed database, for example, due to a disaster in the city where the database is located causing a network outage, that database can initiate a data recovery request. Upon receiving this data recovery request, that database can be identified as the target database.

[0054] The cluster management module 20 determines a target log copy that matches the target database from at least one of the log copies.

[0055] In this embodiment of the disclosure, after determining the target database, the target shard corresponding to the target database can be determined, and the primary database in the target shard can be identified. Then, the cluster management module can query the log replica corresponding to the primary database in the replica server. If multiple log replicas are found, a target log replica can be determined from among the multiple log replicas. This target log replica can be the most consistent replica, that is, the log replica whose relay log has the highest consistency with the logical log in the primary database.

[0056] For the target database 30, a copy of the target log is retrieved, and a data recovery operation is performed based on the target log copy.

[0057] In this embodiment, a sub-thread can be started based on the local copy data in the target database. After determining the target log copy, the sub-thread can lock the storage location of the target log copy on the copy server and retrieve the target log copy based on that location. Then, a storage path can be created in the target database, and the target log copy can be stored in the created storage path. Next, the target log copy can be replayed to achieve data recovery in the target database.

[0058] As described above, in this embodiment of the data recovery system, a replica server, a cluster management module, and a target database are included. The replica server can obtain the logical logs of the master database corresponding to the target database in the distributed database and generate at least one log replica based on the logical logs. Next, the cluster management module can determine a target log replica that matches the target database from among the at least one log replica. Then, the target database can pull the target log replica and perform data recovery operations based on it. This eliminates the need for data transfer between the master database and the target database; instead, data recovery is performed directly based on the logical logs, reducing data transfer time and the risk of data loss.

[0059] In an optional implementation, the cluster management module 20 further includes a cluster manager 21 and a database monitoring agent 22.

[0060] The cluster manager 21 determines the GTID query instruction based on the master database.

[0061] The database monitoring agent 22 determines the GTID of each log replica based on the GTID query command, identifies the log replica whose GTID value meets the query conditions, and designates the log replica as the target log replica.

[0062] In this embodiment of the disclosure, to improve the success rate of obtaining the target log copy in the event of a failure, the generated log copy can be backed up, resulting in multiple log copies. However, during the operation of the main database, when updating the log copies, synchronization differences may exist between different log copies, leading to inconsistencies in content.

[0063] Based on this, a target log replica can be determined among multiple log replicas. This target log replica can be the most consistent replica, that is, the log replica whose relay log has the highest consistency with the logical log in the main database.

[0064] In practical implementation, the first step is to determine the GTID (Global Transaction Identifier) ​​query command through the Cluster Manager (CM). It should be understood that when data is updated in the primary database, the GTID sequence number also increases, and the GTID of the relay logs pulled by the log replicas also increases accordingly. Based on this, the target log replica with the highest consistency can be determined among multiple log replicas by querying the GTID. Here, the CM can be used for replica servers to connect to the primary database, determine the GTID query command, and manage the relay logs in the log replicas.

[0065] Next, the GTID query command mentioned above can be executed through the Database Monitoring Agent (hereinafter referred to as DBAgent). DBAgent can connect to the database and execute the corresponding statements, then return the execution results to the CM. DBAgent can query all log replicas corresponding to the primary database based on the GTID query command, determine the log replica corresponding to the maximum GTID value, and identify that log replica as the target log replica.

[0066] In this embodiment of the disclosure, when there are multiple log replicas corresponding to the primary database, the log replica corresponding to the maximum GTID value can be determined and the log replica is determined as the target log replica, thereby ensuring that the determined target log replica is the most consistent replica, that is, the log replica with the highest consistency between the relay log and the logical log in the primary database.

[0067] In an optional implementation, the cluster management module 20 is further configured to: after determining the GTID of each log replica, determine a target GTID value, determine the log data corresponding to the target GTID value, and determine a clearing instruction based on the log data, wherein the clearing instruction is used to instruct the database monitoring agent to clear the corresponding log data in each log replica.

[0068] In this embodiment of the disclosure, considering that when the logical log in the master data is updated, the relay log pulled by the log replica will also be updated, which will cause the size of the log replica to gradually expand, the relay log in the log replica can be cleaned up to ensure its lightweight nature.

[0069] In practice, log replicas can be cleaned up according to the shard granularity. Here, the target shard corresponding to the target database can be determined, and the relay logs in the log replicas corresponding to the primary database in the target shard can be cleaned up.

[0070] like Figure 3The diagram illustrates the process of cleaning up the log replicas corresponding to the target shard. First, the database nodes (DNs) of the slave databases within the target shard are periodically retrieved. These DNs can include: a `Retrieved_Gtid_Set` (the set of GTIDs received by the slave database from the master database) and an `Executed_Gtid_Set` (the set of GTIDs that have been executed by the slave database). Next, the minimum intersection of the GTID values ​​in this DN, i.e., the `gtidset`, is determined, and this `gtidset` is identified as the target GTID value. It should be understood that the log data corresponding to this target GTID value consists of all data synchronized from all slave databases within the target shard. If any slave database in the target shard fails, it is unnecessary to synchronize this log data from the master database again.

[0071] Next, the target GTID value can be returned to the CM, which can then issue a cleanup request to the replica server to clean up the corresponding log replica. This cleanup request always carries the target GTID value. Then, the DBAgent can execute the cleanup command, specifically the `purge relay logs before 'gtid_set'` command, to clean up the log data corresponding to the target GTID value in the corresponding log replica and return the execution result to the CM.

[0072] In this embodiment of the disclosure, considering that when the logical log in the master data is updated, the relay log pulled by the log replica will also be updated, which will cause the size of the log replica to gradually expand, the relay log in the log replica can be cleaned up to ensure its lightweight nature. Specifically, the log data corresponding to the target GTID value can be cleaned up to ensure that no data is lost when the database is restored in the event of a failure.

[0073] In an optional implementation, the database monitoring agent 22 is further configured to: when the replica server is determined to be the primary database, instruct other servers in the distributed database to perform data processing operations based on the determined target log replica.

[0074] In this embodiment of the disclosure, in a distributed database, when the primary database fails, failover can be performed to switch the secondary database to the primary database. Normally, when selecting a primary database from a secondary database, the replica server does not have the power to elect the primary database. However, when the primary database election rule is set to disallow automatic switching to the same city, the replica server can be used as a temporary primary database, and a data replenishment strategy can be executed through this replica server.

[0075] In practice, the target log replica can be identified first, and the corresponding slave database can be instructed to synchronize data based on the target log replica. Here, the DN corresponding to the target log replica can be determined first, and all slave databases corresponding to the target log replica can be instructed to connect to the DN to synchronize data. After the data synchronization is completed, a new slave database can be selected locally as the master database for switching.

[0076] In this embodiment of the disclosure, when the primary database fails, considering that there may be a situation where the primary election rule is set to disallow automatic switching to the same city, the replica server can be used as a temporary primary server to instruct the corresponding slave server to perform data synchronization, thereby ensuring the efficiency of data recovery when a failure occurs.

[0077] In an optional implementation, the cluster management module 20 is further configured to: after receiving the master database switch instruction, determine the log file sequence number of the log replica, and determine a reset instruction based on the log file sequence number, wherein the reset instruction is used to indicate that the log file sequence number is used as the starting sequence number when resetting the log replica.

[0078] In the embodiments of this disclosure, after a master database switch occurs in the distributed database, the relay log is typically reconfigured and its file sequence number is reset, causing the GTID value of the relay log to be reset accordingly, for example, the GTID value is set to zero. However, in this disclosure, both determining the target log replica and cleaning up the log replica rely on the relay log GTID of the log replica.

[0079] Based on this, upon receiving the primary database switchover command, the log file sequence number of each log replica can be obtained first, and a reset command can be determined based on this log file sequence number. Next, the log replicas can be reset based on this reset command. Specifically, the log file sequence number in the reset command can be used as the starting sequence number, so that subsequent GTID values ​​are incremented based on the log file sequence number.

[0080] In this embodiment of the disclosure, considering that after a master database switchover occurs in a distributed database, the relay log is typically reconfigured and its file sequence number is reset, causing the GTID value of the relay log to be reset accordingly, for example, the GTID value is set to zero. Based on this, upon receiving a master database switchover instruction, this disclosure can determine the log file sequence number of the log replica and determine a reset instruction based on the log file sequence number, so that the subsequent GTID value is incremented based on the log file sequence number. This provides a technical basis for determining the target log replica and performing log replica cleanup.

[0081] In an optional implementation, the data recovery system further includes a data protection device 40;

[0082] When data is lost in the replica server, the data protection device 40 responds to a log recovery command and performs a recovery operation on the log replica in the replica server.

[0083] In this embodiment of the disclosure, a replica server can be installed in the aforementioned data protection device, which can be a protection device capable of withstanding major disasters. In the event of a major disaster, the data protection device can be used to ensure the recoverability of internal data.

[0084] like Figure 4 The diagram shown is an architecture diagram of the data protection device, which includes: a network card interface 41, a log processing module 42, a storage device 43, a storage interface 44, a communication module 45, and a UPS (Uninterruptible Power System) module 46.

[0085] Specifically, network interface card 41 can communicate with the database in the distributed database for log transmission. Considering redundancy, the network interface card can be bonded. Log processing module 42 provides OS capabilities and can mount the storage device as a file system, providing a basic operating environment. It is responsible for log splitting, cleaning up expired log files, and handling log reading, uploading, and downloading services. Storage device 43 serves as the log storage medium, has RAID capabilities, and can perform real-time data flushing.

[0086] Additionally, storage interface 44 allows direct copying of log files and supports network interface access, as well as data transfer via USB, STAT, PCIe, and .m2 interfaces. Communication module 45 can establish wireless communication with external systems after a disaster, enabling data transfer only after connection is established. This module also provides client upload capabilities or can build download services, and includes authentication capabilities. UPS module 46 can provide uninterrupted power for an extended period after a disaster, and can provide power for a corresponding duration based on the uplink speed of the communication module.

[0087] In this embodiment of the disclosure, a replica server can be installed in the aforementioned data protection device to ensure the recoverability of data in the replica server in the event of a major disaster.

[0088] In an optional implementation, the target database 30 is further configured to: determine the log location corresponding to the target log copy, create a storage path, retrieve the target log copy based on the storage path, and perform log replay operations based on the target log copy.

[0089] In this embodiment of the disclosure, a sub-thread can be started based on the local copy data in the target database. After the target log copy is determined, the sub-thread can lock the log position of the target log copy stored in the copy server and pull the target log copy based on the log position.

[0090] Next, a storage path can be created in the target database, and the target log copy can be stored in the created storage path. Then, the relay logs in this target log copy can be replayed, thereby achieving data recovery from the target database.

[0091] In this embodiment of the disclosure, the target log copy may only store relay logs and not data or logical log files, thereby saving storage space and data transmission time during data recovery, greatly improving log copy performance and ensuring maximum lightweighting of the log copy.

[0092] Furthermore, to facilitate understanding of this embodiment, this disclosure also provides a database data recovery method, the execution subject of which is generally a computer device with a certain computing power. In some possible implementations, the database data recovery method can be implemented by a processor calling computer-readable instructions stored in memory.

[0093] See Figure 5 The diagram shows a flowchart of a database data recovery method provided in this embodiment of the present disclosure. The method includes steps S501 to S505, wherein:

[0094] S501: Obtain the logical log of the master database corresponding to the target database in the distributed database, and generate at least one log replica based on the logical log.

[0095] S503: Determine the target log copy that matches the target database from the at least one log copy.

[0096] S505: Retrieve the target log copy and perform data recovery operations in the target database based on the target log copy.

[0097] In this embodiment of the disclosure, the specific implementation method of the database data recovery method is as described above. Figure 1 The corresponding embodiments are described above, and will not be repeated here.

[0098] In summary, in this embodiment of the disclosure, the database data recovery system includes a replica server, a cluster management module, and a target database. The replica server can obtain the logical logs of the master database corresponding to the target database in the distributed database and generate at least one log replica based on the logical logs. Next, the cluster management module can determine a target log replica that matches the target database from among the at least one log replica. Then, the target database can pull the target log replica and perform data recovery operations based on it. This eliminates the need for data transfer between the master and target databases; instead, data recovery is performed directly based on the logical logs, reducing data transfer time and the risk of data loss.

[0099] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0100] Based on the same inventive concept, this disclosure also provides a database data recovery device corresponding to the database data recovery method. Since the principle of the device in this disclosure for solving the problem is similar to the database data recovery method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0101] Reference Figure 6 The diagram shown is a schematic of a database data recovery device according to an embodiment of this disclosure. The device includes: an acquisition unit 61, a determination unit 62, and a recovery unit 63; wherein,

[0102] The acquisition unit 61 is used to acquire the logical log of the master database corresponding to the target database in the distributed database, and generate at least one log copy based on the logical log;

[0103] Determining unit 62 is configured to determine, among the at least one log copy, a target log copy that matches the target database;

[0104] Recovery unit 63 is used to pull the target log copy and perform data recovery operations in the target database based on the target log copy.

[0105] In this embodiment, the database data recovery system includes a replica server, a cluster management module, and a target database. The replica server can obtain the logical logs of the master database corresponding to the target database in the distributed database and generate at least one log replica based on the logical logs. Next, the cluster management module can determine a target log replica that matches the target database from among the at least one log replica. Then, the target database can pull the target log replica and perform data recovery operations based on it. This eliminates the need for data transfer between the master and target databases; instead, data recovery is performed directly based on the logical logs, reducing data transfer time and the risk of data loss.

[0106] In one possible implementation, the determining unit 62 is further configured to:

[0107] Based on the main database, determine the GTID query instruction;

[0108] Based on the GTID query command, the GTID of each log replica is determined, and the log replica whose GTID value meets the query conditions is identified and designated as the target log replica.

[0109] In one possible implementation, the determining unit 62 is further configured to:

[0110] After determining the GTID of each log replica, a target GTID value is determined, and the log data corresponding to the target GTID value is determined. Based on the log data, a clearing instruction is determined, wherein the clearing instruction is used to instruct the clearing of the corresponding log data in each log replica.

[0111] In one possible implementation, the determining unit 62 is further configured to:

[0112] When the replica server is determined as the primary database, other servers in the distributed database are instructed to perform data processing operations based on the determined target log replica.

[0113] In one possible implementation, the determining unit 62 is further configured to:

[0114] Upon receiving the primary database switchover instruction, the log file sequence number of the log replica is determined, and a reset instruction is determined based on the log file sequence number. The reset instruction is used to instruct that the log file sequence number be used as the starting sequence number when resetting the log replica.

[0115] In one possible implementation, the device is also used for:

[0116] When data is lost in the replica server, a log replica in the replica server is restored in response to a log recovery command.

[0117] In one possible implementation, the recovery unit 63 is further configured to:

[0118] Determine the log location corresponding to the target log copy, create a storage path, pull the target log copy based on the storage path, and perform log replay operation based on the target log copy.

[0119] The description of the processing flow of each unit in the device and the interaction flow between each unit can be found in the relevant descriptions in the above method embodiments, and will not be detailed here.

[0120] Corresponding to Figure 1 This disclosure also provides a computer device 700 for data recovery from a database, as described in the data recovery method for the database. Figure 7 The diagram shown is a structural schematic of a computer device 700 provided in an embodiment of this disclosure, including:

[0121] The computer device 700 comprises a processor 71, a memory 72, and a bus 73. The memory 72 stores execution instructions and includes main memory 721 and external memory 722. The main memory 721, also called internal memory, temporarily stores the processing data in the processor 71, as well as data exchanged with external memory such as a hard disk. The processor 71 exchanges data with the external memory 722 through the main memory 721. When the computer device 700 is running, the processor 71 communicates with the memory 72 through the bus 73, causing the processor 71 to execute the following instructions:

[0122] Obtain the logical log of the master database corresponding to the target database in the distributed database, and generate at least one log replica based on the logical log;

[0123] Among the at least one log copy, a target log copy that matches the target database is identified;

[0124] Pull the target log copy and perform data recovery operations in the target database based on the target log copy.

[0125] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the database data recovery method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0126] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the database data recovery method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0127] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0128] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0131] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A data recovery system for a database, applied to a distributed database, characterized in that, include: The replica server, cluster management module, and target database; The replica server obtains the logical logs of the primary database corresponding to the target database in the distributed database, and generates at least one log replica based on the logical logs; The cluster management module identifies a target log replica that matches the target database from at least one of the log replicas; For the target database, retrieve the target log copy and perform data recovery operations based on the target log copy; The step of pulling the target log copy includes: starting a sub-thread based on the local copy data in the target database, and locking the storage location of the target log copy in the copy server through the sub-thread; The target log copy is retrieved based on the storage location.

2. The system according to claim 1, characterized in that, The cluster management module also includes: a cluster manager and a database monitoring agent; The cluster manager determines the GTID query instruction based on the master database; The database monitoring agent determines the GTID of each log replica based on the GTID query command, identifies the log replica whose GTID value meets the query conditions, and designates the log replica as the target log replica.

3. The system according to claim 2, characterized in that, The cluster management module is further configured to: after determining the GTID of each log replica, determine the target GTID value, determine the log data corresponding to the target GTID value, and determine a clearing instruction based on the log data, wherein the clearing instruction is used to instruct the database monitoring agent to clear the corresponding log data in each log replica.

4. The system according to claim 2, characterized in that, The database monitoring agent is also used to: when the replica server is determined to be the primary database, instruct other servers in the distributed database to perform data processing operations based on the determined target log replica.

5. The system according to claim 1, characterized in that, The cluster management module is further configured to: upon receiving a master database switch instruction, determine the log file sequence number of the log replica, and determine a reset instruction based on the log file sequence number, wherein the reset instruction is used to instruct that the log file sequence number be used as the starting sequence number when resetting the log replica.

6. The system according to claim 1, characterized in that, The data recovery system also includes: a data protection device; When data is lost in the replica server, the data protection device, in response to a log recovery command, performs a recovery operation on the log replica in the replica server.

7. The system according to claim 1, characterized in that, The target database is further configured to: determine the log location corresponding to the target log replica, create a storage path, retrieve the target log replica based on the storage path, and perform log replay operations based on the target log replica.

8. A data recovery method for a database, applied to a distributed database, characterized in that, include: Obtain the logical log of the master database corresponding to the target database in the distributed database, and generate at least one log replica based on the logical log; Among the at least one log copy, a target log copy that matches the target database is identified; Pull the target log copy and perform data recovery operations in the target database based on the target log copy; The step of pulling the target log copy includes: starting a sub-thread based on the local copy data in the target database, and locking the storage location of the target log copy in the copy server through the sub-thread; The target log copy is retrieved based on the storage location.

9. A data recovery device for a database, characterized in that, include: The acquisition unit is used to acquire the logical logs of the master database corresponding to the target database in the distributed database, and generate at least one log copy based on the logical logs; A determining unit is configured to determine, among the at least one log copy, a target log copy that matches the target database; The recovery unit is used to pull the target log copy and perform data recovery operations in the target database based on the target log copy; The step of pulling the target log copy includes: starting a sub-thread based on the local copy data in the target database, and locking the storage location of the target log copy in the copy server through the sub-thread; The target log copy is retrieved based on the storage location.

10. A computer device, characterized in that, include: The system includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the database data recovery method as described in claim 8 are performed.

Images