Database fault recovery method based on check points in airborne embedded environment

Abstract

The invention discloses a database fault recovery method based on check points in an airborne embedded environment, which solves the problem that data of a database is correctly persisted in a disk and the database can be correctly and quickly recovered to a state before abnormal closing after a fault occurs. Before a fault occurs, the check point does not consume too many resources during execution, and it is guaranteed that all transactions are not interrupted and executed concurrently. According to the method, the recovery process is simplified, complex processing such as revocation of unsubmitted transactions is avoided, the fault recovery speed and efficiency are improved, and the method is particularly suitable for an embedded environment with high real-time performance requirements.

Description

Technical Field

[0001] The invention belongs to the technical field of databases, and in particular relates to a database failure recovery method based on checkpoints in an airborne embedded environment. Background Art

[0002] The function of the fault recovery module in the airborne embedded database is crucial. First of all, fault recovery is a key component to ensure system reliability. Through an effective fault recovery mechanism, the system can quickly recover and continue to operate normally when a fault occurs, reducing system downtime and improving system stability and reliability. Secondly, in systems involving data, fault recovery is crucial to protecting the integrity of data. Through mechanisms such as transaction logs and checkpoints, the system can quickly recover to a consistent state when a fault occurs to avoid data loss or corruption. Finally, from the perspective of real-time, user experience, and cost-effectiveness in an airborne environment, extremely high requirements are placed on the functional integrity and stability of fault recovery. Therefore, there must be an efficient airborne data fault recovery method, and checkpoints are an important means to improve the efficiency of the fault recovery method.

[0003] The checkpoint-based fault recovery method in general databases is based on the Redo+Undo log and write-ahead log method. However, in an airborne embedded environment, the airborne database has the following characteristics:

[0004] 1. In general databases, the buffer pool adopts the Steal+No Force strategy, which allows dirty data to be written back to disk before the transaction is committed, but does not force dirty data to be written back to disk immediately when the transaction is committed. This strategy delays the time to write data back to disk, reduces disk I / O operations, improves system performance and throughput, and maintains data consistency and reliability. However, in airborne embedded databases, the No Steal+No Force strategy is adopted. The No Steal strategy does not allow dirty data to be written to disk before the transaction is committed, and the buffer pool retains the state of the data before modification. When a transaction fails, it only needs to perform a rollback operation in the buffer pool, and there is no need to perform an undo operation on the disk to roll back, thereby improving the efficiency of transaction failure processing.

[0005] 2. In general database management systems, there is usually a dedicated thread responsible for periodically checking the current database status and deciding whether to start a checkpoint operation. This thread exists to ensure the persistence and consistency of the database and trigger a checkpoint when necessary to write the data in memory to the disk. However, in an airborne embedded environment, the database component is usually embedded in the application. Due to resource constraints and real-time requirements, there is no dedicated thread responsible for periodically starting a checkpoint operation.

[0006] 3. Redo+Undo logs are generally used in general databases. These two logs are needed to recover data during fault recovery. Undo logs are used to roll back transactions when transactions fail. However, in airborne databases, for real-time considerations, in-memory shadow pages are used to roll back transactions, and Undo logs are abandoned. Therefore, only Redo logs are used in airborne embedded databases, and traditional fault recovery methods cannot be directly used for fault recovery in airborne environments.

[0007] In summary, the checkpoint mechanism based on write-ahead log and Redo+Undo log cannot be directly used for fault recovery in airborne databases due to problems such as poor real-time performance, no independent thread to execute checkpoints, and incompatibility of fault recovery modules. Summary of the invention

[0008] In order to overcome the shortcomings of the prior art, the present invention provides a database fault recovery method based on checkpoints in an airborne embedded environment, which solves the problem of correctly persisting the data of the database in the disk, and can correctly and quickly recover to the state before the abnormal closure after the database fails; before the failure occurs, the checkpoint execution will not consume too many resources, and ensure that each transaction is not interrupted and executed concurrently. The present invention simplifies the recovery process, avoids complex processing such as the revocation of uncommitted transactions, improves the speed and efficiency of fault recovery, and is particularly suitable for embedded environments with high real-time requirements.

[0009] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0010] Step 1: Buffer pool strategy;

[0011] The onboard embedded database adopts the No Steal+No Force buffer pool strategy. A control block PCN is designed for each data page in the buffer pool. The PCN stores the address of the old page before the corresponding page is modified and the address of the new page after the buffer pool is modified. If the old page exists in the PCN, it means that the page is being modified; if it does not exist, it means that the page is not used by the transaction.

[0012] Step 2: Write the log before writing data;

[0013] Before making actual changes to the database, the corresponding log records are written to the log file; any changes to the database disk data must first be written to the log, and the data modifications are temporarily stored in the memory buffer, and the database modified data is persisted to the disk at a later time; each change operation of each transaction, such as insert, update, delete, etc., will generate log records, and these log records together constitute a transaction log sequence;

[0014] Step 3: Checkpoint in log;

[0015] The log types include BEGIN, COMMIT, ABORT, UPDATE, and CHECKPOINT, which represent the logs of transaction start, commit, rollback, update, and checkpoint respectively. Whenever a transaction is created and started, a BEGIN log is written to the log file. The modification of data during the transaction execution is written to the UPDATE type log. Finally, according to the completion status of the transaction, a COMMIT log is written, indicating the completion of the transaction, or an ABORT log is written, indicating the rollback of the transaction. The checkpoint log represents the start and end of the checkpoint execution of the database system. Each log has the number LSN, log type type, and log size size information. The log file records the above five types of log information.

[0016] When the log entries in the log file reach a certain threshold, the checkpoint mechanism is triggered to write the checkpoint log. The flag bit of the checkpoint log is "0", indicating that the checkpoint mechanism has started. After the checkpoint is completed, that is, the dirty page is written to the disk, the checkpoint log with the flag bit "1" is written again, indicating that the checkpoint execution of this stage is completed.

[0017] Step 4: Checkpoint-based fault recovery;

[0018] When a system failure occurs, the onboard embedded database will check whether the log file exists when it is restarted. If it does not exist, it means that the database has been shut down normally and the log file has been deleted. If it exists, it means that fault recovery steps are required. There are many checkpoint logs in the log, but the committed transaction modification operations between every two complete checkpoints have been written to the disk, so only the committed transaction modification content after the last complete checkpoint needs to be restored.

[0019] Fault recovery is divided into three steps:

[0020] a) Search for the first complete checkpoint from the end of the log file, that is, a pair of checkpoints with flags "0" and "1". The data to be recovered is the committed transaction content after the checkpoint with flag "0".

[0021] b) Successfully committed transactions need to be recovered, while uncommitted or rolled back transactions do not need to be recovered or undone. Therefore, COMMIT and BEGIN type logs are searched from the end of the log file and their LSN numbers are recorded. In subsequent traversals, UPDATE logs that need to be recovered are searched. If the LSN of the UPDATE log matches the LSN of the COMMIT and BEGIN type logs, it means that the data needs to be recovered.

[0022] c) Different transactions may modify the same data multiple times, but in the recovery step, only the last modified data needs to be restored, that is, the last modified data content includes all the previously modified contents of the data; during the recovery process, the log file is traversed from back to front, and the UPDATE logs that meet the conditions are recorded and restored. There is no need to restore the UPDATE logs of the same data page number later.

[0023] A computer program enables a computer to execute the above database failure recovery method.

[0024] An electronic device comprises: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above-mentioned database failure recovery method.

[0025] A computer-readable storage medium stores a computer program, which implements the above-mentioned database failure recovery method when executed by a processor.

[0026] A chip includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes the above-mentioned database failure recovery method.

[0027] A computer program product comprises a computer storage medium storing a computer program, wherein the computer program comprises instructions executable by at least one processor, and the above-mentioned database failure recovery method is implemented when the instructions are executed by the at least one processor.

[0028] The beneficial effects of the present invention are as follows:

[0029] 1) Designed for airborne embedded environments. The data structure is simple and clear, with no additional overhead to increase system burden, and no need to add additional thread monitoring.

[0030] 2) Small storage space occupation: Because lightweight logs are used, only Redo is used, and Undo is not required. Therefore, when recovering from a failure, the system only needs to record the modification operations of the committed transactions. This method significantly reduces the size of the log file and reduces storage requirements. It is more suitable for embedded environments with limited resources and helps improve the overall efficiency and response speed of the system.

[0031] 3) High fault recovery efficiency:

[0032] ① With checkpoints, only the most recent checkpoint state needs to be restored. This means that after a failure, the system only needs to restore the modifications of transactions that have been committed since the last complete checkpoint, which significantly reduces the amount of data that needs to be processed.

[0033] ② No undo operation is required during recovery, thus simplifying the recovery process. Since only committed transactions need to be redone, the system can be restored to a consistent state more quickly, avoiding complex processing such as undoing uncommitted transactions. This design improves the speed and efficiency of fault recovery, and is particularly suitable for embedded environments with high real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of the PageControlNode structure;

[0035] Figure 2 This is a schematic diagram of the relationship between pageControlMap, flushMap, txFlushMap, and oldPageData;

[0036] Figure 3 Design a schematic diagram for the check log;

[0037] Figure 4 Write a flow diagram for the checkpoint;

[0038] Figure 5 Design a schematic diagram for database logging;

[0039] Figure 6 Write a specific implementation process for the checkpoint;

[0040] Figure 7 The following is a diagram of the steps for data recovery after a failure occurs. DETAILED DESCRIPTION

[0041] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0042] The purpose of the present invention is to efficiently restore data in the airborne database in the event of a system failure, thereby ensuring the integrity and consistency of the data. A log and failure recovery method based on checkpoints in an airborne environment is specifically provided.

[0043] The problem to be solved by the present invention is to solve the problem that the data of the database is correctly persisted in the disk, and the database can be correctly and quickly restored to the state before the abnormal shutdown after a failure occurs. Before the failure occurs, the checkpoint execution will not consume too many resources, and ensure that each transaction is not interrupted and executed concurrently.

[0044] The technical solution adopted by the present invention to solve the technical problem includes the following contents:

[0045] 1. Buffer pool strategy. The onboard embedded database adopts the No Steal + No Force buffer pool strategy. No Steal means that some memory resources need to be sacrificed. The No Steal strategy will not write the dirty pages of uncommitted transactions back to the disk, and the No Force strategy avoids disk write operations when each transaction is committed. A control block Page Control Node (PCN) is designed for each data page in the buffer pool. The PCN stores the address of the old page before the corresponding page is modified and the address of the new page after the current buffer pool modification. If the old page exists in the PCN, it means that the page is being modified. PCN is used to concurrently write the backup pages of uncommitted transactions through checkpoints, which improves the concurrent efficiency of transactions. If the number of data pages is large, the efficiency of finding dirty pages by traversing is too low, so designing the dirty page to be flushed linked list FlushMap can significantly improve the efficiency of flushing dirty pages.

[0046] 2. Write logs before data writing operations. The core idea of ​​Write-Ahead Logging (WAL) is to write corresponding log records to the log file before making actual changes to the database. Any changes to the database disk data must first be written to the log, and the data modifications are temporarily stored in the memory buffer. The database modified data is persisted to the disk at a later time. Each change operation of each transaction, such as insert, update, delete, etc., will generate log records, and these log records together constitute a transaction log sequence.

[0047] Although log records are first written to the memory buffer, they are not immediately written to persistent storage (disk). Instead, log records are flushed to disk in batches under certain conditions (such as transaction commit or buffer fill) to balance performance and reliability. When a transaction commits, the state of the transaction is marked as "committed", which means that all log records related to the transaction have been persisted. This step ensures that in the event of a system crash or failure, the database can recover the state of the transaction through the log.

[0048] 3. Checkpoints in the log. The log types include BEGIN, COMMIT, ABORT, UPDATE, and CHECKPOINT, which represent the logs of transaction start, commit, rollback, update, and checkpoint, respectively. Whenever a transaction is created and started, a BEGIN log is written to the log file. The modification of data during the transaction execution is written to the UPDATE type log. Finally, the COMMIT (transaction completion) or ABORT (transaction rollback) log is written according to the completion status of the transaction. The checkpoint log represents the start and end of the checkpoint execution of the database system. Each log has information such as number (LSN), log type (type), and log size (size). The log file records the above five types of log information. When the log entries in the log file reach a certain threshold, the checkpoint mechanism is triggered to write the checkpoint log. The checkpoint log flag bit is "0", indicating that the checkpoint mechanism has started; after the checkpoint is completed, that is, the dirty page is written to the disk, a checkpoint log with a flag bit of "1" is written, indicating that the checkpoint execution of this stage is completed.

[0049] 4. Checkpoint-based fault recovery. When a system failure occurs, the onboard embedded database will check whether the log file exists when it is restarted. If it does not exist, it means that the database has been shut down normally and the log file has been deleted. If it exists, it means that fault recovery steps are required. There are many checkpoint logs in the log, but the committed transaction modification operations between every two complete checkpoints have been written to the disk, so only the committed transaction modification content after the last complete checkpoint needs to be restored. Fault recovery can be roughly divided into the following three steps:

[0050] a) Search forward from the end of the log file for the first complete checkpoint, that is, a pair of checkpoints with flags "0" and "1". The data to be recovered is the committed transaction content after the checkpoint with flag "0".

[0051] b) Successfully committed transactions need to be recovered, while uncommitted or rolled back transactions do not need to be recovered or undone. Therefore, search for COMMIT and BEGIN type logs from the end of the log file and record their numbers (LSN). In subsequent traversals, search for UPDATE logs that need to be recovered. If their LSNs match the above LSNs, it means that the data needs to be recovered.

[0052] c) Different transactions may modify the same data multiple times, but in the recovery step, only the last modified data needs to be restored, because the log records the result of the data modification, not the specific data operation, that is, the last modified data content includes all the previously modified contents of the data. During the recovery process, the log file is traversed from back to front, and the UPDATE logs that meet the conditions are recorded and restored. There is no need to restore the UPDATE logs of the same data page number later.

[0053] Example:

[0054] Database failure recovery can be divided into two modules: before the failure occurs and after the failure occurs. The function before the failure occurs is mainly to store the data of the committed transactions in the log file, which needs to ensure the concurrency between transactions and the orderliness of data storage; after the failure occurs, the data sequence that needs to be recovered is found in the log to recover the data. The following is a detailed explanation of these two aspects.

[0055] 1. Before the failure occurs;

[0056] a) PCN design:

[0057] In a database system, a data file is divided into a series of fixed-size blocks, which are called "data pages". The data pages store specific data content. The present invention designs a PageControlNode (PCN) structure to manage the mutually exclusive access of each data page. Figure 1 As shown, tid represents the most recent transaction id of the page managed by the PCN that was modified. Since the database does not use an Undo log, if a transaction is rolled back, it can only be rolled back based on the backup page in the buffer pool instead of relying on the Undo log. Therefore, the PCN needs to store the backup page before the page is modified, that is, oldPage. If a transaction wants to modify the page, the data of the page needs to be backed up to the oldPage field in preparation for rollback. When a transaction is committed normally, the oldPage field of all pages modified by the transaction is set to NULL. In the present invention, if oldPage is NULL, it means that the page has no backup, that is, the page is not being modified by a transaction, so as to judge the status of the page when executing a checkpoint. newPage permanently points to the page in the buffer pool, regardless of whether the page is being modified. Latch is a mutual exclusion lock of the PCN, which ensures that the PCN can only be accessed by one transaction or checkpoint at the same time, thereby ensuring data consistency and avoiding data errors.

[0058] PCN not only stores pageControlMap, but also stores txFlushMap and flushMap. The relationship between the three is as follows Figure 2As shown. Among them, oldPageData stores the backup page of the page and is placed in the transaction structure. The other three are HashMaps, with key as pid and value as PCN. pageControlMap stores the PCN of all pages in the buffer pool. Before a transaction modifies the page, it must query the container to obtain the control access right of the PCN. flushMap represents the queue of pages to be flushed of the checkpoint. Whenever a transaction is committed, the PCN corresponding to the page modified by the transaction must be added to the flushMap, indicating that these pages have been modified and can be refreshed to the disk. txFlushMap is set in each transaction structure to store the PCN modified by the current transaction, so that when the transaction is committed or rolled back, txFlushMap can be accessed in time to obtain the access control right of the PCN and modify its status.

[0059] flushMap is mainly used for checkpoint traversal access to PCN, and txFlushMap is mainly used for transaction traversal access to PCN. The purpose of designing these two additional HashMaps is to share the access volume of pageControlMap and improve the accuracy of PCN access. If only relying on a single pageControlMap, the checkpoint will spend more time looking for the PCN to be refreshed and the transaction will spend more time looking for the PCN modified by the transaction, resulting in lower concurrency performance between multiple transactions.

[0060] b) Checkpoint write:

[0061] When a transaction completes writing a log, if the number of logs in the file reaches the set threshold (the default is 5000), a checkpoint is triggered. When a checkpoint is triggered, a checkpoint start log is first written to the log. The format of the checkpoint log is as follows: Figure 3 As shown in the figure, status is set to "0" or "1", which indicates the start and end of the checkpoint, respectively. The checkpoint is then executed, the PCN in the flushMap is traversed, the dirty pages in the buffer pool are obtained and written to the disk. When the checkpoint is completed, the checkpoint end log is written to the log again, that is, the checkpoint log with status set to "1" is written. The checkpoint writing steps are as follows: Figure 4 A log file diagram after a complete checkpoint execution is shown in Figure 5 shown.

[0062] The checkpoint writing process mentioned above is shown in Figure 6 ;

[0063] 2. After the failure occurs;

[0064] Reference Figure 7 ,The data recovery steps after a failure are as follows.

[0065] Step 1: The algorithm performs necessary variable initialization to prepare for the recovery process. Then, two lists, commitList and redoPageIdList, are created. commitList is used to store transactions that have been confirmed to be committed, while redoPageIdList is used to record the page IDs that have been recovered.

[0066] Step 2, the algorithm then checks whether the log file is successfully copied to the temporary file and tries to rename it. If this process is unsuccessful, it returns directly and ends the recovery process. After that, a log buffer is created for reading and processing log entries.

[0067] Step 3, in the main loop, the algorithm first reads the smallest unit of complete log. If the size of the read log is equal to the size of the BEGIN, COMMIT or ABORT log, the log type is determined. For a COMMIT type log, if it occurs after the checkpoint, it will be added to the commitList; for an ABORT type log, if the BEGIN log exists in the commitList, it will be removed from the list.

[0068] Step 4: If the log size is equal to the size of the UPDATE log, read the complete UPDATE log again and check its type. If the page ID of the log already exists in redoPageIdList, it indicates that the page has been flushed to disk. Otherwise, determine whether it is a redo log based on the information in commitList. If so, write the page to the database data file for subsequent loading.

[0069] Step 5: For the processing of checkpoint logs, the algorithm will first determine the log type, filter out incomplete checkpoints, and set a sentinel variable after finding the checkpoint. This sentinel variable is used to mark the position of the checkpoint to ensure that the algorithm only processes log entries after the checkpoint. Finally, if the seek variable (indicating the current read position) is less than or equal to 0, it indicates that the log file has been read and the algorithm ends.

Claims

1. A database failure recovery method based on checkpoints in an airborne embedded environment, characterized in that: The steps include: Step 1: Buffer pool strategy; The onboard embedded database adopts the No Steal+No Force buffer pool strategy. A control block PCN is designed for each data page in the buffer pool. The PCN stores the address of the old page before the corresponding page is modified and the address of the new page after the buffer pool is modified. If the old page exists in the PCN, it means that the page is being modified; if it does not exist, it means that the page is not used by the transaction. Step 2: Write the log before writing data; Before making actual changes to the database, the corresponding log records are written to the log file; any changes to the database disk data must first be written to the log, and the data modifications are temporarily stored in the memory buffer, and the database modified data is persisted to the disk at a later time; each change operation of each transaction, such as insert, update, delete, etc., will generate log records, and these log records together constitute a transaction log sequence; Step 3: Checkpoint in log; The log types include BEGIN, COMMIT, ABORT, UPDATE, and CHECKPOINT, which represent the logs of transaction start, commit, rollback, update, and checkpoint respectively. Whenever a transaction is created and started, a BEGIN log is written to the log file. The modification of data during the transaction execution is written to the UPDATE type log. Finally, according to the completion status of the transaction, a COMMIT log is written, indicating the completion of the transaction, or an ABORT log is written, indicating the rollback of the transaction. The checkpoint log represents the start and end of the checkpoint execution of the database system. Each log has the number LSN, log type type, and log size size information. The log file records the above five types of log information. When the log entries in the log file reach a certain threshold, the checkpoint mechanism is triggered to write the checkpoint log. The checkpoint log flag is "0", indicating that the checkpoint mechanism has started; After the checkpoint is completed, that is, the dirty page is written to the disk, a checkpoint log with a flag bit of "1" is written to indicate that the checkpoint at this stage is completed. Step 4: Checkpoint-based fault recovery; When a system failure occurs, the onboard embedded database will check whether the log file exists when it is restarted. If it does not exist, it means that the database is closed normally and the log file has been deleted. If it exists, it means that the fault recovery steps need to be performed; There are many checkpoint logs in the log, but the committed transaction modification operations between every two full checkpoints have been written to the disk, so only the committed transaction modifications after the last full checkpoint need to be restored; Fault recovery is divided into three steps: a) Search for the first complete checkpoint from the end of the log file, that is, a pair of checkpoints with flags "0" and "1". The data to be recovered is the committed transaction content after the checkpoint with flag "0". b) Successfully committed transactions need to be recovered, while uncommitted or rolled back transactions do not need to be recovered or undone. Therefore, COMMIT and BEGIN type logs are searched from the end of the log file and their LSN numbers are recorded. In subsequent traversals, UPDATE logs that need to be recovered are searched. If the LSN of the UPDATE log matches the LSN of the COMMIT and BEGIN type logs, it means that the data needs to be recovered. c) Different transactions may modify the same data multiple times, but in the recovery step, only the last modified data needs to be restored, that is, the last modified data content includes all the previously modified contents of the data; During the recovery process, the log files are traversed from back to front, and the UPDATE logs that meet the conditions are recorded and recovered. There is no need to recover the UPDATE logs with the same data page number later.

2. A computer program, characterized in that The computer program enables a computer to execute the method as claimed in claim 1.

3. An electronic device, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as claimed in claim 1.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method as claimed in claim 1 is implemented.

5. A chip, characterized in that: include: A processor, used to call and run a computer program from a memory, so that a device equipped with the chip executes the method as claimed in claim 1.

6. A computer program product, characterized in that The computer program product comprises a computer storage medium storing a computer program, wherein the computer program comprises instructions executable by at least one processor, and when the instructions are executed by the at least one processor, the method according to claim 1 is implemented.

Images