Database fault recovery method and device, equipment, medium and program product

Abstract

The embodiment of the invention provides a database fault recovery method and device, equipment, a medium and a program product. The database system comprises a main database and a standby database, the method is applied to the main database, and the method comprises the following steps: analyzing a pre-write log file of the main database, and determining information of a changed first data block in the main database; based on the information of the first data block, generating a bitmap of a data file where the first data block is located, the bitmap being used for recording whether each data block in the data file changes; and in response to a fault recovery request from the standby database, sending the first data block to the standby database based on the bitmap, so that the standby database performs fault recovery based on the first data block. The method is used for improving the recovery efficiency of the fault standby database.

Description

Technical Field

[0001] This application relates to the field of database recovery, and in particular to a database fault recovery method, apparatus, equipment, medium, and program product. Background Technology

[0002] In a database read-write separation cluster architecture, when the backup database fails due to hardware failure, network interruption, or storage anomaly, it must be restored to its data synchronization state with the primary database as soon as possible to ensure the redundancy of the cluster and business continuity.

[0003] In existing technologies, when a backup database fails and cannot be recovered via the Rewind (rollback recovery) mechanism, a full copy of the primary database data directory or a basic backup performed using tools is typically used for redoing. This method relies on full data transfer during the recovery process, resulting in high network bandwidth usage and long recovery times.

[0004] Therefore, how to shorten the recovery time of the fault-standby database and improve the recovery efficiency is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a database fault recovery method, apparatus, device, medium, and program product to improve the efficiency of fault backup database recovery.

[0006] In a first aspect, embodiments of this application provide a database fault recovery method. The database system includes: a primary database and a backup database. The method is applied to the primary database and includes:

[0007] Parse the write-ahead log file of the main database to determine the information of the first data block in the main database that has changed;

[0008] Based on the information of the first data block, a bitmap of the data file containing the first data block is generated. The bitmap is used to record whether each data block in the data file has changed.

[0009] In response to a fault recovery request from the backup database, the first data block is sent to the backup database based on the bitmap, so that the backup database can perform fault recovery based on the first data block.

[0010] In one possible implementation, generating a bitmap of the data file containing the first data block based on the information of the first data block includes:

[0011] Based on the information of the first data block, determine the target location of the first data block in the data file;

[0012] In the bitmap of the data file, the bit at the target location is marked as a first identifier; the bits in the bitmap other than the target location are marked as a second identifier.

[0013] In one possible implementation, in the memory of the main database, multiple bitmaps are organized into chains using a Least Recently Used (LRU) approach. Before marking the bit at the target location in the bitmap of the data file with a first identifier, the method further includes:

[0014] In response to the fact that the number of the plurality of bitmaps in the chain is equal to a preset number and there is no bitmap of the data file in the chain, the bitmap at the end of the chain is cleared to obtain the cleared bitmap;

[0015] The cleared bitmap is used as the bitmap of the data file.

[0016] In one possible implementation, sending the first data block to the backup database based on the bitmap includes:

[0017] The information recorded in the bitmap is stored in a bitmap file on the disk of the main database, and the bitmap is then cleared.

[0018] Based on the bitmap file, the first data block is sent to the backup database.

[0019] In one possible implementation, after clearing the bitmap, the method further includes:

[0020] When the bitmap is updated, the first log sequence number (LSN) of the write-ahead log is recorded;

[0021] If no anomalies are detected in the backup database, and the second LSN of the replay write-ahead log of the backup database is greater than the first LSN, the bitmap file in the disk is cleared.

[0022] In one possible implementation, the backup database is also used for:

[0023] Obtain the first data directory information of the main database;

[0024] The first data directory information is compared with the second data directory information of the backup database to obtain the comparison result;

[0025] Based on the comparison results, perform at least one of the following operations:

[0026] Delete redundant data directory information from the backup database; the redundant data directory information is data directory information that does not exist in the first data directory information.

[0027] Create missing data directory information in the backup database; the missing data directory information is data directory information that exists in the first data directory information but does not exist in the second data directory information.

[0028] Secondly, embodiments of this application provide a database fault recovery device. The database system includes: a primary database and a backup database. The device is applied to the primary database and includes:

[0029] The determination module is used to parse the write-ahead log file of the main database and determine the information of the first data block that has changed in the main database;

[0030] The generation module is used to generate a bitmap of the data file containing the first data block based on the information of the first data block. The bitmap is used to record whether each data block in the data file has changed.

[0031] The sending module is configured to, in response to a fault recovery request from the backup database, send the first data block to the backup database based on the bitmap, so that the backup database can perform fault recovery based on the first data block.

[0032] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0033] The memory stores computer-executed instructions;

[0034] The processor executes computer execution instructions stored in the memory, causing the processor to perform the method described in any of the first aspects above.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in any of the first aspects above.

[0036] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the first aspects above.

[0037] This application provides a database fault recovery method, apparatus, device, medium, and program product. It determines the information of the first changed data block by parsing the write-ahead log file of the primary database and generates a bitmap recording the changes in the data block. Upon receiving a fault recovery request from a standby database, it sends the first data block to the standby database according to the bitmap to assist in its recovery. Compared to the traditional method of full copying, which has a long recovery time and consumes high network bandwidth when dealing with large amounts of data, this method uses incremental recovery, transmitting only the first changed data block. This significantly reduces the amount of data transmitted during the recovery process, thereby accelerating the recovery speed of the faulty standby database, reducing the pressure on network bandwidth, and improving the efficiency and resource utilization of database fault recovery. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0039] Figure 1 A schematic flowchart illustrating a database fault recovery method provided in an embodiment of this application;

[0040] Figure 2 This is a schematic diagram of a bitmap logic structure provided in an embodiment of this application;

[0041] Figure 3 A schematic diagram of a bitmap marker provided in an embodiment of this application;

[0042] Figure 4 A schematic diagram of a bitmap LRU chain provided in an embodiment of this application;

[0043] Figure 5 This application provides a schematic diagram of a process for recording changed data block information in a master database.

[0044] Figure 6 A schematic diagram illustrating a backup database recovery process provided in an embodiment of this application;

[0045] Figure 7 This application provides a schematic diagram of the structure of a database fault recovery device.

[0046] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application.

[0047] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0048] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0049] In this application, the term "comprising" and its variations can refer to non-limiting inclusion; the term "or" and its variations can refer to "and / or". The terms "first", "second", etc., in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0050] In a database read-write splitting cluster architecture, the primary database handles write operations and maintains data consistency, while the backup database replicates data changes from the primary database in real time to achieve read-write splitting, load balancing, and high availability. If the backup database fails due to hardware failure, network interruption, or storage anomalies, it must be restored to its data synchronization state with the primary database as quickly as possible to ensure cluster redundancy and business continuity.

[0051] However, existing technologies for backup database failure recovery have significant drawbacks: when the backup database fails and cannot be recovered via the Rewind (rollback recovery) mechanism, a full copy of the primary database data directory is typically performed, or a basic backup is executed using the sys_basebackup (a system basic backup tool) tool. In large-scale data scenarios (such as 900GB databases), such methods can take more than 10 hours to recover, and consume significant network bandwidth during this process.

[0052] Furthermore, existing recovery mechanisms directly discard unchanged data blocks in the failover database, resulting in the repeated transmission of a large amount of valid data. This wastes storage resources and increases the complexity of the recovery process. This method may cause serious problems such as business interruption and service delays. Therefore, there is an urgent need for a failover database recovery method that can reduce network transmission volume and shorten recovery time to meet the operational needs of large-scale database clusters.

[0053] Therefore, this embodiment of the application determines the information of the first data block that has changed by parsing the write-ahead log file of the primary database and generates a bitmap recording the changes in the data block. When a backup database failure recovery request is received, the first data block is sent to the backup database according to the bitmap to assist in its recovery. Compared with the traditional method of full copying, which has a long recovery time and consumes high network bandwidth when the data volume is large, this method uses incremental recovery, transmitting only the first data block that has changed, which greatly reduces the amount of data transmitted during the recovery process, thereby speeding up the recovery speed of the backup database, reducing the pressure on network bandwidth, and improving the efficiency and resource utilization of database failure recovery.

[0054] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0055] It should be noted that the executing entity of this application can be any electronic device with processing capabilities, such as a user terminal or a server. For example, it can be an electronic device that runs a main database management system or executes a specific database operation program.

[0056] Figure 1 This is a flowchart illustrating a database fault recovery method provided in an embodiment of this application, as shown below. Figure 1 As shown, this method is applied to the main database, and the method includes:

[0057] S101. Parse the write-ahead log file of the main database to determine the information of the first data block that has changed in the main database.

[0058] Optionally, the master database can be a database for data storage and processing tasks. It is the main source of data and is responsible for handling various data operation requests, such as data insertion, updating, and deletion, to ensure data integrity and consistency.

[0059] Write-ahead logging (WAL) is a database logging method. Before data modification operations are executed, relevant modification information is recorded in the log file. This way, even if the database system encounters a failure during data modification, it can be restored to its pre-failure state through the log file, ensuring data durability and consistency.

[0060] The first data block can be a data block that has changed in the main database. A data block can be the basic unit for storing data in a database. Information about the first data block can include one or more of the following: its location, size, and the data it contains.

[0061] Alternatively, the electronic device can employ existing methods for analyzing and processing write-ahead log files of the main database. By reading the information recorded in the log files, it can extract content related to the data that has changed in the main database. Based on the information obtained from parsing the log files, it can determine the specific information of the first data block that has changed in the main database, such as the location, size, and data content of the data block, or any one or more of these.

[0062] S102. Based on the information of the first data block, generate a bitmap of the data file where the first data block is located. The bitmap is used to record whether each data block in the data file has changed.

[0063] Optionally, a data file can be a file in the main database used to actually store the data. The data is organized and stored in the data file in the form of data blocks, and a data file can include multiple data blocks. A bitmap can be a data structure composed of a series of binary bits, with each bit corresponding to a data block in the data file, and the value of the bit indicating whether the object has changed. Figure 2 This is a schematic diagram of a bitmap logic structure provided in an embodiment of this application, such as... Figure 2 As shown, the initial state of a bitmap can be represented by 0.

[0064] Optionally, the electronic device can create a bitmap of the data file containing the first data block based on the information of the first data block. That is, based on information such as the position of the first data block in the data file, the correspondence between each bit in the bitmap and the data block in the data file is determined, and the values ​​of the corresponding bits are set to indicate whether the data block has changed.

[0065] In one implementation, the electronic device can first determine the target location of the first data block in the data file based on the information of the first data block. Then, the bits at the target location in the bitmap of the data file are marked as a first identifier; the bits in the bitmap other than the target location are marked as a second identifier.

[0066] Optionally, the target location of the first data block in the data file can be the specific location occupied by the first data block in the data file. Since the data file consists of multiple data blocks, each data block has its specific storage location, and the target location clearly defines the exact location of the first data block in the data file.

[0067] For example, the first identifier can be represented by a binary value of 1, indicating that the first data block corresponding to the target position has changed. The second identifier can be represented by a binary value of 0, indicating that the data blocks corresponding to the bits in the bitmap other than the target position have not changed. Figure 3 This is a schematic diagram of a bitmap marker provided in an embodiment of this application, such as... Figure 3As shown, if the third block in the data file (shown as a file) changes, its target position in the corresponding bitmap is marked as the first identifier, i.e., 1.

[0068] Optionally, the electronic device can accurately locate the target position of the first data block in the data file by analyzing the information of the first data block. If the first data block has changed, the bit at the target position is marked as a first identifier in the bitmap of the data file; if none of the other data blocks in the data file have changed, the electronic device can mark the bits in the bitmap other than the target position as a second identifier. This marking method can characterize whether each data block in the data file has changed.

[0069] In this embodiment of the application, when generating the bitmap, the target location of the first data block in the data file is first determined, and then the bit at the target location is marked as a first identifier, while the remaining bits are marked as a second identifier. This method can accurately record the changes of each data block in the data file. Through simple bit identifiers, it is possible to quickly determine which data blocks have changed. During subsequent fault recovery, the data blocks that need to be transmitted can be accurately selected, avoiding unnecessary data transmission and further improving the accuracy and efficiency of database fault recovery, making the fault recovery process more efficient and reliable.

[0070] S103. In response to a fault recovery request from the standby database, a first data block is sent to the standby database based on a bitmap, so that the standby database can perform fault recovery based on the first data block.

[0071] Optionally, a standby database can serve as a backup to the primary database, taking over its functions in the event of a primary database failure to ensure the normal operation of the database system. The standby database will maintain data synchronization with the primary database to minimize data loss during fault recovery.

[0072] When the backup database fails, fault recovery is required to maintain data synchronization with the primary database. In this case, the backup database can send a fault recovery request to the primary database, requesting the necessary data to restore it to normal operating status. Optionally, electronic devices can receive and process the fault recovery request from the backup database via an interface call.

[0073] Optionally, the electronic device can, based on the generated bitmap, transfer the first data block marked as changed in the bitmap from the primary database to the backup database, enabling the backup database to obtain the changed data from the primary database. The backup database can then perform fault recovery operations based on this data, updating its own data to maintain consistency with the primary database. Optionally, this transfer process can employ any one or more existing database synchronization mechanisms, such as streaming replication, incremental log transmission, etc.

[0074] In one implementation, the electronic device can store the information recorded in the bitmap to a bitmap file on the disk of the main database and then clear the bitmap. Next, based on the bitmap file, a first data block is sent to a backup database.

[0075] Optionally, the main database disk can persistently store the bitmap to a bitmap file. This bitmap file stores the bitmap as a file, and can be created if the bitmap file does not exist on the main database disk. Alternatively, the electronic device can use existing technology to write the bitmap to the bitmap file on the main database disk to complete the storage of bitmap record information.

[0076] Optionally, the electronic device can set all bits in the bitmap to their initial state (e.g., 0), so that it no longer contains any valid marker information, thereby clearing the bitmap. Optionally, the electronic device can employ one or more database synchronization mechanisms, such as streaming replication or incremental log transmission, to send data blocks to a standby database to ensure consistency between the primary and standby databases.

[0077] This application embodiment can store bitmap information in a disk bitmap file, which on the one hand prevents the bitmap information from being lost due to memory failure, ensuring data persistence; on the other hand, clearing the bitmap frees up memory space for recording changes in new data blocks later. Transmitting the first data block based on the bitmap file on the disk makes data transmission more stable and reliable, not limited by memory, ensuring the smooth progress of the fault recovery process and improving the stability and reliability of the entire database fault recovery system.

[0078] This application embodiment determines the information of the first changed data block by parsing the write-ahead log file of the primary database and generates a bitmap recording the changes in the data block. Upon receiving a backup database failure recovery request, the first data block is sent to the backup database according to the bitmap to assist in its recovery. Compared to the traditional method of full copying, which has a long recovery time and consumes high network bandwidth when the data volume is large, this method uses incremental recovery, transmitting only the first changed data block, which greatly reduces the amount of data transmitted during the recovery process. This speeds up the recovery of the backup database, reduces the pressure on network bandwidth, and improves the efficiency and resource utilization of database failure recovery.

[0079] In one implementation, in the memory of the main database, multiple bitmaps are organized into a chain using the Least Recently Used (LRU) method. Before the electronic device marks the bit at the target position in the bitmap of the data file as the first identifier, the electronic device can also, in response to the fact that the number of multiple bitmaps in the chain is equal to a preset number and there is no bitmap of the data file in the chain, clear the bitmap at the end of the chain to obtain the cleared bitmap, and use the cleared bitmap as the bitmap of the data file. Figure 4 A schematic diagram of a bitmap LRU chain provided in an embodiment of this application is shown below. Figure 4 As shown, when the number of multiple bitmaps in the chain is equal to the preset number, and there are no bitmaps of data files in the chain, the bitmap at the end of the chain can be cleared.

[0080] Optionally, the main database's memory can be used for temporary data storage. Compared to persistent storage devices such as disks, memory offers faster read and write speeds, improving the database system's operational efficiency. LRU can be a cache eviction algorithm; when memory is full and space needs to be freed up for new data, the least recently used data is evicted first. In this embodiment, multiple bitmaps are organized into a chain using LRU. The bitmaps form a linked list structure according to their most recently accessed order, with the most recently accessed bitmap near the head and the least recently accessed bitmap near the tail.

[0081] The preset quantity can be pre-stored in the electronic device; it can be a pre-set value used to limit the maximum number of bitmaps in the chain. For example, the preset quantity could be 10.

[0082] Alternatively, the electronic device can set all bits in the bitmap to an initial state (e.g., 0), so that it no longer contains any valid marker information, thus clearing the bitmap. After clearing the bitmap, it can be reused to record new state information as a bitmap for a data file.

[0083] In this embodiment, the main database memory organizes multiple bitmaps into a chain using an LRU (Least Recently Used) approach. When the number of bitmaps in the chain equals a preset number and no bitmap exists for the data file, the tail bitmap of the chain is cleared and used as the bitmap for that data file. This method effectively manages bitmap resources in memory, preventing memory from being exhausted due to storing too many bitmaps and ensuring efficient memory utilization. It can operate stably when processing changes in a large number of data blocks, ensuring the accuracy and timeliness of bitmap records and providing a reliable basis for subsequent fault recovery.

[0084] After clearing the bitmap as described above, the electronic device can also record the first log sequence number (LSN) of the write-ahead log when the bitmap is updated. Secondly, if no anomalies are detected in the standby database, and the second LSN of the standby database's replay write-ahead log is greater than the first LSN, the bitmap file on the disk is cleared.

[0085] Optionally, the log sequence number is used to uniquely identify each log record in the WAL, and is an incrementing number used to track the log write order and synchronization progress. The first LSN can be the sequence number of the latest log in the current WAL recorded by the primary database, reflecting the latest update status of the primary database. The second LSN can be the sequence number of the latest log in the standby database after replaying the WAL log, reflecting the synchronization progress of the standby database.

[0086] Optionally, the electronic device can write the first LSN of the current WAL to disk during bitmap updates. The electronic device can periodically check the running status and LSN synchronization of the standby database; for example, one or more indicators such as a normal standby database connection and no error logs can suggest that the standby database is functioning correctly. When the second LSN of the standby database is greater than the first LSN of the primary database, it indicates that the standby database has completed all updates to the primary database. At this point, the bitmap file on the disk can be deleted, or its contents can be reset to clear the bitmap file on the disk.

[0087] In this embodiment, after clearing the bitmap, the first log sequence number (LSN) of the WAL is recorded during bitmap updates. When it is detected that the backup database is normal and the second LSN of the replayed WAL is greater than the first LSN, the bitmap file in the disk is cleared, which avoids the bitmap file occupying disk space for a long time, saves disk resources, and also ensures the simplicity and efficiency of the system.

[0088] Based on the above embodiments, after the standby database sends a fault recovery request to the primary database, it can obtain the first data directory information of the primary database and compare the first data directory information with the second data directory information of the standby database to obtain the comparison result.

[0089] Optionally, the first data directory information may be the directory structure of the primary database storing data files, and any one or more of the metadata, etc. For example, it may include any one or more of the following: file list, path, size, checksum, etc. This first data directory information can reflect the current data storage status of the primary database. The second data directory may be the directory structure of the standby database storing data files, and any one or more of the following of the metadata, etc., and it should correspond to the first data directory information.

[0090] Optionally, the electronic device can use built-in database tools or a custom script to compare the first data directory information with the second data directory information of the backup database to obtain the comparison results. The comparison results may include any one or more of the following: missing files, such as the backup database lacking files from the primary database; inconsistent files, such as mismatched file size, modification time, or checksum; and path differences, such as different directory structures (e.g., the primary database uses symbolic links, while the backup database is not synchronized).

[0091] Based on the comparison results, the standby database can perform one or more operations, such as deleting redundant data directory information or creating missing data directory information. Redundant data directory information refers to data directory information that does not exist in the primary data directory information, which may be generated due to temporary files or logs not being properly cleaned up during the synchronization process. Missing data directory information refers to data directory information that exists in the primary data directory information but does not exist in the secondary data directory information, which may be due to data being added to the primary database but not successfully synchronized to the standby database.

[0092] Optionally, the standby database can also copy missing files from the primary database, and can also truncate any extra files in the standby database compared to the primary database.

[0093] This application embodiment can delete redundant or create missing data directory information based on the results. Compared to traditional fault recovery methods that ignore data directory consistency, leading to mismatches between the standby database and the primary database, this application embodiment ensures that the data directory structure of the standby database is completely consistent with that of the primary database, providing a solid foundation for subsequent fault recovery based on the first data block.

[0094] In summary, the embodiments of this application are mainly divided into two parts: one part is that the primary database starts a background process to record the block changes in the database, and the other part is the recovery logic of the backup database based on the block change information. These two parts will be described in detail below.

[0095] The main logic for changes to data blocks in the primary database is as follows:

[0096] 1. Set the standby host name in the standby_hosts parameter (new parameter) of the master database configuration file. If the parameter is empty, the function is disabled and the background process for recording block changes will not be started.

[0097] 2. When the master database starts, a background process is created. This process is responsible for parsing the WAL log from the WAL file in real time and recording the changed data block information.

[0098] (1) The background process can hold a maximum of 10 bitmaps in memory by default, and periodically refreshes the bitmap information to the corresponding bitmap file (if it does not exist, it creates the bitmap file), and then sets the bitmap to empty.

[0099] (2) When the WAL log parsing finds that a data block in a data file has occurred, the corresponding bit in the bitmap is marked as 1.

[0100] (3) The bitmaps in memory are organized into a chain using the LRU method. When a data file changes, there are already 10 bitmaps in the chain, and there is no bitmap corresponding to the file in the chain, the bitmap information at the end of the chain will be refreshed to the bitmap file, and then the file will be used after the chain is cleared.

[0101] (4) A horizontal line LSN is set in memory. After each refresh of all bitmaps to the bitmap file, it is cleared. The corresponding LSN of the WAL log is recorded when any bitmap is updated for the first time after the refresh.

[0102] 3. The background process periodically checks the standby database connection status. If all connections are normal, the previously recorded block change information is cleared; otherwise, the recorded block change information is retained.

[0103] (1) If the detection finds that all standby databases are in normal status and all standby databases replay WAL logs are greater than the horizontal line LSN, then clear all bitmap files (note that bitmaps in memory are not cleared).

[0104] (2) If the detection finds that there is an abnormality in the backup database status, then do nothing and continue to retain all text and image information.

[0105] Figure 5 This application provides a schematic diagram of a process for recording changed data block information in a master database, as shown in the embodiments of this application. Figure 5 As shown, the process includes:

[0106] 1. Start the main database.

[0107] 2. Start the background collection process.

[0108] 3. Parse the WAL logs in the WAL file.

[0109] 4. Record block change information to the local map.

[0110] 5. Determine if the horizontal line is empty. If yes, proceed to step 6; otherwise, proceed to step 7.

[0111] 6. Set the horizontal line LSN to the LSN used to parse the WAL log.

[0112] 7. Determine if Timer 1 has timed out. Timer 1 is used to monitor the standby database status check interval. If yes, proceed to step 8; otherwise, proceed to step 9.

[0113] 8. Determine if the standby database is in normal status. If yes, proceed to step 12; otherwise, proceed to step 3.

[0114] 9. Determine if timer 2 has timed out. Timer 2 controls the time period for writing bitmap information to disk (i.e., refreshing to disk). If yes, proceed to step 10; otherwise, proceed to step 3.

[0115] 10. Bitmap information is written to disk.

[0116] 11. Clear the memory bitmap and horizontal line LSN, and return to step 3.

[0117] 12. Clear the bitmap file and return to step 3.

[0118] Main logic for backup database failure recovery:

[0119] Figure 6 This is a schematic diagram of a backup database recovery process provided in an embodiment of this application, such as... Figure 6 As shown, the process includes:

[0120] 1. Establish connection with the primary database and initiate a backup database failure recovery request.

[0121] 2. After receiving the request, the master database executes sys_basebackup_start.

[0122] 3. Bitmap information is written to disk, and the background process refreshes all bitmaps in memory to the bitmap file.

[0123] 4. Retrieve the primary database data directory information, compare it with the faulty database directory, and perform the following operations:

[0124] (1) Delete redundant files and directories in the backup database.

[0125] (2) Create a backup database missing directory.

[0126] (3) Copy the missing files from the primary database to the backup database.

[0127] (4) Truncate redundant content in the backup database file.

[0128] 5. The primary database reads the bitmap file and sends the changed data blocks (i.e., the marked data blocks) to the backup database. The backup database then fills in or replaces the corresponding data blocks in the file.

[0129] 6. Determine if data block transmission has ended. If yes, proceed to step 7; otherwise, proceed to step 5.

[0130] 7. Copy the primary database's WAL file to the backup database.

[0131] 8. The main database executes sys_basebackup_stop.

[0132] 9. Start the backup database.

[0133] 10. Backup database restoration complete.

[0134] This application embodiment employs an incremental approach to restore the faulty standby database. It parses the WAL file in the primary database, records data block change logic using a bitmap, and incrementally restores the standby database using this data block change information. Compared to existing technologies that do not reuse valuable data in the faulty database directory, where the faulty data directory is discarded during recovery (and much of the data in the faulty data directory remains unchanged in the primary database), this application embodiment uses an incremental approach to restore the faulty database. By reusing data in the faulty database that has not changed in the primary database, it significantly reduces the amount of data transmitted during the recovery process, accelerates the recovery speed of the standby database, and reduces network bandwidth pressure.

[0135] The above are the method embodiments provided in this application. The apparatus provided in this application will be described below.

[0136] Figure 7 This application provides a schematic diagram of the structure of a database fault recovery device, as shown below. Figure 7 As shown, the database fault recovery device 400 provided in this embodiment includes: a determining module 401, a generating module 402, and a sending module 403. Optionally, the database fault recovery device 400 may further include a processing module 404.

[0137] Module 401 is used to parse the write-ahead log file of the main database and determine the information of the first data block that has changed in the main database.

[0138] The generation module 402 is used to generate a bitmap of the data file containing the first data block based on the information of the first data block. The bitmap is used to record whether each data block in the data file has changed.

[0139] The sending module 403 is used to send a first data block to the standby database based on a bitmap in response to a fault recovery request from the standby database, so that the standby database can perform fault recovery based on the first data block.

[0140] Optionally, the generation module 402 is specifically used to determine the target location of the first data block in the data file based on the information of the first data block. The bits at the target location in the bitmap of the data file are marked as a first identifier; the bits in the bitmap other than the target location are marked as a second identifier.

[0141] For example, in the memory of the main database, multiple bitmaps are organized into a chain using the Least Recently Used (LRU) method. Before the generation module 402 marks the bit at the target position in the bitmap of the data file as the first identifier, the processing module 404, in response to the condition that the number of multiple bitmaps in the chain equals a preset number and there are no bitmaps of the data file in the chain, clears the bitmap at the tail of the chain, obtaining a cleared bitmap. The cleared bitmap is then used as the bitmap of the data file.

[0142] Optionally, the sending module 403 is specifically used to store the information recorded in the bitmap to a bitmap file on the disk of the main database, and then clear the bitmap. Based on the bitmap file, the first data block is sent to the backup database.

[0143] For example, after the sending module 403 is specifically used to clear the bitmap, the processing module 404 is further used to record the first log sequence number (LSN) of the write-ahead log when the bitmap is updated. If no anomalies are detected in the standby database, and the second LSN of the standby database's replay write-ahead log is greater than the first LSN, the bitmap file on the disk is cleared.

[0144] Optionally, the backup database is also used to: obtain the first data directory information of the primary database.

[0145] The first data directory information is compared with the second data directory information in the backup database to obtain a comparison result. Based on the comparison result, at least one of the following operations is performed: Delete redundant data directory information from the backup database; the redundant data directory information is data directory information that does not exist in the first data directory information. Create missing data directory information in the backup database; the missing data directory information is data directory information that exists in the first data directory information but does not exist in the second data directory information.

[0146] The database fault recovery device provided in this embodiment can execute the methods provided in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0147] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 8 As shown, the electronic device 500 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 500 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0148] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0149] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0150] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0151] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0152] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0153] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0154] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0155] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0156] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0157] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0158] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0160] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0161] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0162] Finally, it should be noted that other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from its scope.

Claims

1. A database fault recovery method, characterized in that, The database system includes a primary database and a standby database. The method is applied to the primary database, and the method includes: Parse the write-ahead log file of the main database to determine the information of the first data block in the main database that has changed; Based on the information of the first data block, a bitmap of the data file containing the first data block is generated. The bitmap is used to record whether each data block in the data file has changed. In response to a fault recovery request from the backup database, the first data block is sent to the backup database based on the bitmap, so that the backup database can perform fault recovery based on the first data block.

2. The method according to claim 1, characterized in that, The step of generating a bitmap of the data file containing the first data block based on the information of the first data block includes: Based on the information of the first data block, determine the target location of the first data block in the data file; In the bitmap of the data file, the bit at the target location is marked as a first identifier; the bits in the bitmap other than the target location are marked as a second identifier.

3. The method according to claim 2, characterized in that, In the memory of the main database, multiple bitmaps are organized into chains using the Least Recently Used (LRU) method. Before marking the bit at the target location in the bitmap of the data file as the first identifier, the method further includes: In response to the fact that the number of the plurality of bitmaps in the chain is equal to a preset number and there is no bitmap of the data file in the chain, the bitmap at the end of the chain is cleared to obtain the cleared bitmap; The cleared bitmap is used as the bitmap of the data file.

4. The method according to any one of claims 1-3, characterized in that, The step of sending the first data block to the backup database based on the bitmap includes: The information recorded in the bitmap is stored in a bitmap file on the disk of the main database, and the bitmap is then cleared. Based on the bitmap file, the first data block is sent to the backup database.

5. The method according to claim 4, characterized in that, After clearing the bitmap, the method further includes: When the bitmap is updated, the first log sequence number (LSN) of the write-ahead log is recorded; If no anomalies are detected in the backup database, and the second LSN of the replay write-ahead log of the backup database is greater than the first LSN, the bitmap file in the disk is cleared.

6. The method according to any one of claims 1-3, characterized in that, The backup database is also used for: Obtain the first data directory information of the main database; The first data directory information is compared with the second data directory information of the backup database to obtain the comparison result; Based on the comparison results, perform at least one of the following operations: Delete redundant data directory information from the backup database; the redundant data directory information is data directory information that does not exist in the first data directory information. Create the missing data directory information in the backup database; The missing data directory information refers to data directory information that exists in the first data directory information but does not exist in the second data directory information.

7. A database fault recovery device, characterized in that, The database system includes: a primary database and a backup database, the device being used with the primary database, the device comprising: The determination module is used to parse the write-ahead log file of the main database and determine the information of the first data block that has changed in the main database; The generation module is used to generate a bitmap of the data file containing the first data block based on the information of the first data block. The bitmap is used to record whether each data block in the data file has changed. The sending module is configured to, in response to a fault recovery request from the backup database, send the first data block to the backup database based on the bitmap, so that the backup database can perform fault recovery based on the first data block.

8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images