An embedded database transaction processing method, device, equipment and storage medium

Abstract

The application discloses an embedded database transaction processing method, device and equipment and a storage medium, and the method comprises the following steps: based on a database write transaction request, a new data page is allocated for the transaction in the database and writing is completed on the data page; an identifier of the transaction and an allocated data page resource list are recorded to generate an intent log; in response to a preset trigger condition, the transaction on the data page is batched to the database; if an exception occurs in the landing process, page resource recycling is performed by using the data page resource list recorded in the intent log to restore the complete database. The application can meet the high-performance operation of the database on one hand, and can guarantee the integrity and availability of the database file when the system is abnormal, thereby effectively avoiding the risk of database file damage.

Description

Technical Field

[0001] This invention relates to the field of embedded database processing technology, and in particular to an embedded database transaction processing method, apparatus, device, and storage medium. Background Technology

[0002] Embedded databases are widely used in various network devices (such as OTN, PTN, switches, and data communication equipment) to store critical business data such as configuration information and status data. In these application scenarios, embedded databases not only need to meet the extremely low latency requirements of high concurrency and write-intensive scenarios, but also must ensure data integrity and consistency in the face of abnormal situations such as unexpected power outages and system crashes. However, existing methods for ensuring the reliability of embedded databases often struggle to balance high performance and high reliability.

[0003] Existing methods for ensuring the reliability of embedded databases include: 1. Write-Ahead Logging (WAL): Before modifying data files, all modifications must be recorded in a log file. In the event of a system crash, data can be recovered through log records. 2. Copy-on-Write (CoW): When data needs to be modified, the original data is not directly overwritten. Instead, a copy of the data is created and modified on the copy. After modification, the entire database is updated via atomic pointer switching. Problems with existing technologies include: 1. Strict transaction operations impact performance: Achieving reliability requires synchronous disk writes (logs or data) within transactions. This becomes a performance bottleneck in high-concurrency, write-intensive scenarios. 2. Abandoning transaction operations does not guarantee reliability: Both technologies provide a fully asynchronous data write method without using transactions, but in the event of system anomalies (such as unexpected power outages), database file corruption and irrecoverability can occur.

[0004] Therefore, how to balance the high performance and reliability of embedded databases is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The main objective of this invention is to provide an embedded database transaction processing method, apparatus, device, and storage medium, which can meet the high-performance operation of the database on the one hand, and ensure the integrity and availability of the database file in the event of system abnormality on the other hand, effectively avoiding the risk of database file corruption.

[0006] In a first aspect, this application provides an embedded database transaction processing method, wherein the method includes the following steps: Based on the database write transaction request, a new data page is allocated for the transaction in the database and the write is completed on the data page; Record the identifier of the transaction and the list of allocated data page resources to generate an intent log; In response to a preset trigger condition, the transactions on the data page are batch-written to disk and then to the database. If an anomaly occurs during the disk write process, the data page resource list recorded in the intent log will be used to reclaim page resources in order to restore the complete database.

[0007] In conjunction with the first aspect mentioned above, as an optional implementation, in response to a crash and restart during the disk write process, the database state of the last complete persistence is used as a benchmark, and the list of data page resources is extracted based on the records of the intent logs read. The page resources corresponding to the data page resource list are recycled to the database's idle page list to restore the database to a consistent, ready-to-use state.

[0008] In conjunction with the first aspect above, as an optional implementation, if a database write transaction commit request is received, a new data page is allocated for the transaction in the memory-mapped area of ​​the database file using a copy-on-write mechanism; Write the data changes corresponding to the transaction to the data page.

[0009] In conjunction with the first aspect mentioned above, as an optional implementation method, it is possible to determine whether the intent log file exists and contains log records; If log records exist and are included, discard all data changes made by transactions in the log records.

[0010] In conjunction with the first aspect mentioned above, as an optional implementation, a disk synchronization operation is performed on the intent log to persist the intent log, and a successful transaction commit is returned to the client without synchronizing the database file.

[0011] In conjunction with the first aspect above, as an optional implementation method, it is determined whether a preset trigger condition is met. The preset trigger condition includes: whether the set timer has timed out, whether the intent log threshold meets the set condition, or whether a processing instruction has been received. If the preset triggering conditions are met, the transactions written on the new data page will be batch synchronized to the database, thus making the current state of the memory-mapped area of ​​the database file persistent.

[0012] In conjunction with the first aspect mentioned above, as an optional implementation method, if it is determined that the transaction has been successfully synchronized to the database, the intent log file is truncated or cleared.

[0013] Secondly, this application provides an embedded database transaction processing apparatus, the apparatus comprising: An allocation module is used to allocate a new data page for the transaction in the database and complete the writing on the data page based on the database write transaction request. A generation module is used to record the identifier of the transaction and the list of allocated data page resources to generate an intent log; An execution module is used to, in response to a preset trigger condition, batch write transactions on the data page to disk and then to the database. The processing module is used to reclaim page resources using the data page resource list recorded in the intent log if an abnormality occurs during the disk write process, so as to restore the complete database.

[0014] Thirdly, this application also provides an electronic device, the electronic device comprising: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement the method described in any one of the first aspects.

[0015] Fourthly, this application also provides a computer-readable storage medium storing computer program instructions that, when executed by a computer, cause the computer to perform the method described in any of the first aspects.

[0016] This application provides an embedded database transaction processing method, apparatus, device, and storage medium. The method includes the following steps: based on a database write transaction request, allocating new data pages for the transaction in the database and completing the write operation on the data pages; recording the identifier of the transaction and a list of allocated data page resources to generate an intent log; responding to a preset trigger condition, batch writing the transactions on the data pages to disk to the database; if an anomaly occurs during the disk writing process, using the data page resource list recorded in the intent log to reclaim page resources and restore the complete database. This application can satisfy high-performance database operation on the one hand, and ensure the integrity and availability of database files in the event of system anomalies on the other hand, effectively avoiding the risk of database file corruption.

[0017] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit the invention. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] Figure 1 This is a flowchart of an embedded database transaction processing method provided in the embodiments of this application; Figure 2 This is a schematic diagram of an embedded database transaction processing device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the lightweight transaction commit provided in the embodiments of this application; Figure 4 This is a schematic diagram of periodic asynchronous checkpoint flushing provided in the embodiments of this application; Figure 5 This is a schematic diagram illustrating the execution of crash recovery operations provided in the embodiments of this application; Figure 6 This is a schematic diagram of the solution provided in the embodiments of this application; Figure 7 This is a schematic diagram of the intent log and main database storage design provided in the embodiments of this application; Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of this application; Figure 9 This is a schematic diagram of a computer-readable program medium provided in an embodiment of this application. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0021] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. Some of the block diagrams shown in the drawings represent functional entities and do not necessarily correspond to physically or logically independent entities.

[0022] The embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0023] Reference Figure 1 , Figure 1 The diagram shown is a flowchart of an embedded database transaction processing method provided by the present invention. Figure 1 As shown, the method includes the following steps: Step S101: Based on the database write transaction request, allocate a new data page for the transaction in the database and complete the write on the data page.

[0024] Specifically, if a database write transaction commit request is received, a new data page is allocated for the transaction in the memory-mapped area of ​​the database file using the copy-on-write mechanism; and the data changes corresponding to the transaction are written to the data page.

[0025] To illustrate, suppose a database system receives a write transaction (e.g., updating data A). The system first allocates one or more new data pages for the current transaction in the database file's memory-mapped area using a copy-on-write (CoW) mechanism. If the existing data page is numbered Page 100, the system does not directly modify the original page but copies it to a new physical location (e.g., Page 100_COW) and points the transaction's write operation record to this new page. Subsequently, the transaction writes the update operation (e.g., changing data A to data B) to the newly allocated data page. Before the transaction commits, the original data page remains unchanged, and other read transactions can still access the old data on Page 100; only at commit time is the mapping in the metadata atomically switched from the original page to the new page (or the page version is marked). If the transaction rolls back, only the CoW page is discarded; there is no need to restore the original page.

[0026] Step S102: Record the identifier of the transaction and the list of allocated data page resources to generate an intent log.

[0027] Specifically, recording the transaction identifier and the list of allocated data page resources to generate an intent log can be understood as follows: after a transaction acquires new data pages through the copy-on-write mechanism, the system records the transaction identifier and the list of allocated data page resources, generating an intent log. This log is persisted to storage before the actual data change occurs, and is used to identify incomplete transactions and their occupied resource copies after a system crash.

[0028] In other words, the intent log contains: append-only entries recording non-persistent database transactions and their associated business IDs. For example: Transaction ID1, {a list of page IDs modified by this transaction}; Transaction ID2, {a list of page IDs modified by this transaction}.

[0029] The characteristics of intent logs are: 1. They only record transaction IDs and their list of page IDs. Compared to recording the transaction operation content, the cost is very small. Most of the content involved in a transaction can be persisted within a single page, resulting in high persistence performance. 2. In some application scenarios, if a separate intent log file is not desired, intent logs can be recorded in a specific page of the database file.

[0030] In one embodiment, after recording the transaction identifier and the list of allocated data page resources to generate an intent log, the process includes: performing a disk synchronization operation on the intent log to persist the intent log, and returning a successful transaction commit to the client without synchronizing the database file. This means constructing a log record containing the transaction identifier and the list of page resources allocated in this transaction; appending this log record to the intent log file; and performing a disk synchronization operation on the intent log file to ensure the record is persisted; and returning a successful transaction commit to the client without performing a disk synchronization operation on the main data file. In other words, only the data corresponding to the transaction is changed in memory, such as changing a specific piece of data on page 11 from A to B, but it is not yet written to the database. The disk write is completed in one step according to preset conditions.

[0031] Step S103: In response to a preset trigger condition, batch write the transactions on the data page to disk and then to the database.

[0032] Specifically, it is determined whether the preset trigger conditions are met. The preset trigger conditions include: whether the set timer has timed out, whether the intent log threshold meets the set conditions, or whether a processing instruction has been received. If the preset trigger conditions are met, the transactions written on the new data page are batch synchronized to the database, so that the current state of the memory mapping area of ​​the database file is persisted.

[0033] To illustrate this with an example, periodic checkpointing is performed as follows: triggered by preset conditions (such as timers or log size thresholds) or external commands, the checkpointing operation includes: a. Locking the system to prevent new non-blocking commits. b. Batch synchronizing transactions written to new data pages to the database, fully persisting the current state of its memory-mapped area (including the result of the most recent non-blocking commit). c. After successful synchronization, truncating or clearing the intent log file.

[0034] It is understandable that periodic checkpoints can be used to synchronize transaction requests to the database in batches. That is, while data pages are allocated in step S101, no data is written to disk in step S101. This step, however, writes all the transactions to be changed in step S101 to the database all at once. In other words, step S101 only writes the changed transaction information to a blank page in memory without updating the database; this step performs a batch update to the database.

[0035] In one embodiment, if it is determined that the transaction has been successfully synchronized to the database, the intent log file is truncated or cleared. It is understood that the intent log file is truncated or cleared after successful batch synchronization.

[0036] Step S104: If an anomaly occurs during the disk write process, the page resources are reclaimed using the data page resource list recorded in the intent log to restore the complete database.

[0037] Specifically, in response to a crash and restart during disk write-to-disk, the system uses the last fully persisted database state as a baseline and determines whether the intent log file exists and contains log records. If it exists and contains log records, all transaction data changes in the log records are discarded. Based on the read intent log records, a data page resource list is extracted. The page resources corresponding to the data page resource list are reclaimed to the database's idle page list to restore the database to a consistent, ready-to-use state.

[0038] For ease of understanding, an example is given: when the system restarts from an unexpected crash, the baseline database state that last successfully completed step S103 in the main data file is loaded. The existence of the intent log file and whether it contains valid records are checked. If valid log records exist, all data change content of transactions recorded in the log is discarded, but the page resource list in all log records is extracted. Based on the baseline database state, all pages in the extracted page resource list are recycled to the database's free page list. The intent log file is cleared, and the service is started based on the cleaned database state.

[0039] Understandably, during system crash recovery, the baseline database state corresponding to the last successful checkpoint is first loaded from the master data file. Next, the intent log file is checked for valid records. If they exist, since the CoW data pages of these transactions were lost before being merged, the system will not attempt to replay the transaction changes. Instead, it will discard the data change content in each log entry and only extract the list of page resources (i.e., the new pages allocated by the transaction). Then, based on the baseline state, all these pages are reclaimed to the database's list of free pages, thus forming a consistent state without any trace of incomplete transactions. Finally, the intent log file is cleared, and the database service is started in this cleaned-up state. This recovery strategy avoids invalid data page references, ensures that free resources are not leaked, and guarantees strong database consistency. It should be explained that anomalies can occur during the disk write process, including: abnormal power outages, device restarts, device crashes, device freezes, etc. Understandably, these anomalies indicate that the device cannot perform the disk write process normally.

[0040] Reference Figure 2 , Figure 2 The diagram shown is a schematic of an embedded database transaction processing device provided by the present invention. Figure 2 As shown, the device includes: The allocation module 201 is used to allocate a new data page for the transaction in the database and complete the writing on the data page based on the database write transaction request. The generation module 202 is used to record the identifier of the transaction and the list of allocated data page resources to generate an intent log; Execution module 203 is used to write transactions on the data page to disk in batches to the database in response to a preset trigger condition; The processing module 204 is used to reclaim page resources by using the data page resource list recorded in the intent log if an abnormality occurs during the disk writing process, so as to restore the complete database.

[0041] Furthermore, in one possible implementation, the processing module is also configured to, in response to a crash and restart during the disk write process, extract a list of data page resources based on the state of the last fully persisted database and on the records of the read intent log. The page resources corresponding to the data page resource list are recycled to the database's idle page list to restore the database to a consistent, ready-to-use state.

[0042] Furthermore, in one possible implementation, the allocation module is also configured to, upon receiving a database write transaction commit request, allocate new data pages for the transaction in the memory-mapped area of ​​the database file using a copy-on-write mechanism; Write the data changes corresponding to the transaction to the data page.

[0043] Furthermore, in one possible implementation, the processing module is also used to determine whether the intent log file exists and contains log records; If log records exist and are included, discard all data changes made by transactions in the log records.

[0044] Furthermore, in one possible implementation, the generation module is also configured to perform a synchronized disk operation on the intent log to persist the intent log and return a successful transaction commit to the client without synchronizing the database file.

[0045] Furthermore, in one possible implementation, the execution module is also used to determine whether a preset trigger condition is met, the preset trigger condition including: whether the set timer has timed out, whether the intent log threshold meets the set condition, or whether a processing instruction has been received; If the preset triggering conditions are met, the transactions written on the new data page will be batch synchronized to the database, thus making the current state of the memory-mapped area of ​​the database file persistent.

[0046] Furthermore, in one possible implementation, the execution module is also used to truncate or clear the intent log file if it is determined that the transaction has been successfully synchronized to the database.

[0047] Reference Figure 3, Figure 3 The diagram shown is a schematic of the lightweight transaction commit provided by the present invention. Figure 3 As shown: Understandably, a transaction involves initiating a write operation and modifying the data, obtaining new data pages in the memory-mapped region through the copy-on-write (CoW) mechanism, and writing the changes to these new pages.

[0048] Intent Log: Before a transaction is committed, the system records the transaction identifier, the list of allocated data page resources (and optional change descriptions), and performs a disk synchronization operation on the intent log to achieve persistence. Then, it can return a successful commit to the client without synchronizing the main database file.

[0049] Understandably, within the memory-mapped area of ​​the database file, new data pages and metadata (file header) are generated using the Copy-on-Write (CoW) mechanism. A log record containing the transaction identifier and a list of page resources allocated in this transaction is constructed. This log record is appended to the intent log file, and a disk synchronization operation is performed on the intent log file to ensure the record is persisted. Without performing a disk synchronization operation on the main data file, a transaction success is returned to the client.

[0050] Reference Figure 4 , Figure 4 The diagram shown is a schematic of the periodic asynchronous checkpoint flushing provided by the present invention. Figure 4 As shown, it includes: Block new transaction commits: Suspend the system from receiving new write transaction commit requests, ensuring that no new intent log entries or CoW page changes are generated during checkpoint execution, thereby maintaining static consistency of the state to be synchronized.

[0051] Perform a full synchronous disk operation on the database: Batch flush transaction changes from all committed but not yet merged into the master database file's CoW data pages to the database file, and call fsync or msync to ensure the current state of the memory-mapped regions is persisted to disk. This step brings the database file to a fully consistent state containing all completed transactions.

[0052] Intent log file clearing: After the database files have been fully synchronized, clear the intent log file (or truncate it). Since all changes from committed transactions have been safely saved to the master database files at this point, the records in the intent log are no longer needed for recovery. The clearing operation prevents the log from growing indefinitely and provides a clean starting point for the next recovery.

[0053] Reopen new transaction commits: Reopen the system to receive new transaction commit requests, and the database resumes normal concurrent processing capabilities.

[0054] It is understandable that periodic checkpointing operations are performed: triggered by preset conditions (such as timers, log size thresholds) or external commands, checkpointing operations are executed, including: a. Lock the system to prevent new non-blocking commits.

[0055] b. Perform a complete synchronization operation (write to disk) on the transaction on the data page, and fully persist the current state of its memory-mapped area (including the result of the most recent non-blocking commit).

[0056] c. After successful synchronization, truncate or clear the intent log file.

[0057] Reference Figure 5 , Figure 5 The diagram shown illustrates the crash recovery operation provided by this invention. Figure 5 As shown, it includes: Understandably, the database state that last completed the checkpoint operation is found in the main database file: the root page is determined.

[0058] The system first loads the database snapshot corresponding to the last successfully completed checkpoint, including identifying the root page, as the baseline consistent state for recovery.

[0059] Check if the intent log contains valid records. If so, extract the list of page resources. The system checks if the intent log file exists and contains valid records. If it exists, extract the list of data page resources allocated to the transaction (i.e., new pages allocated under the copy-on-write mechanism) from each log record, and discard the specific data change content (because the changes have not yet been merged into the master database file and the CoW pages may have been lost).

[0060] Clean up the database state: Based on the page resource list in the intent log, reclaim pages to free pages. Based on the checkpoint baseline state obtained in step 1, reclaim all pages from the extracted page resource list (i.e., unmerged CoW pages) to the database's free page list, thereby cleaning up all traces of incomplete transactions and forming a consistent database state without resource leaks.

[0061] Intent log file clearing: Clearing the intent log file (or truncating it) prevents the same log records from being processed repeatedly during the next recovery. The database then starts normal service based on the cleaned-up state. This entire process ensures that the database can quickly recover to a consistent state even if there were unmerged copy-on-write transactions during the last run, without causing page resource leaks.

[0062] That is, perform crash recovery operations: when the system restarts from an unexpected crash: a. Load the last successfully completed checkpoint operation and consistent baseline database state from the database file.

[0063] b. Check if the intent log file exists and contains valid records.

[0064] c. If valid log records exist, discard all data changes recorded in the logs for all transactions, but extract the list of page resources from all log records.

[0065] d. Based on the baseline database state, all pages in the extracted page resource list are recycled to the database's idle page list, forming a cleaned and consistent database state.

[0066] e. Clear the intent log file and start the service based on the cleaned database state.

[0067] In summary, this application creates an "intent log" that is not for data recovery but only for identifying transactions and recording resource usage. Secondly, it designs a guided rollback cleanup and recovery mechanism that uses the log to repair the resource list of the previous secure version, rather than rolling forward to recover transactions. Thirdly, it explicitly divides database operations into two modes: ultra-fast asynchronous commit and periodic synchronous persistence, using the intent log as a security guarantee.

[0068] Reference Figure 6 , Figure 6 The diagram shown is a schematic representation of the solution provided by the present invention. Figure 6 As shown, it includes: Lightweight transaction commit: ① Generation: When a transaction performs a write operation, it generates new data pages through the copy-on-write (CoW) mechanism and generates corresponding intent log records (including transaction identifiers and a list of allocated data page resources).

[0069] ② Update: The transaction writes the data changes to the new data page and performs a synchronous disk operation (fsync) on the intent log to achieve persistence. Then it can return a successful commit to the client without updating the main database file.

[0070] Periodic asynchronous checkpoint flushing: ③ Usage: When preset trigger conditions are met (such as timer timeout, log threshold reached, or instruction received), the system will batch synchronize the committed transaction changes on multiple new data pages to the main database file (i.e., the resource list recorded in the "Usage" intent log is actually written to disk).

[0071] Perform crash recovery operations: ④ Recovery: Upon system restart after a crash, the system begins by checking the intent log file from the baseline database state of the last checkpoint. If valid records exist, all page resource lists are extracted and recycled to idle pages (discarding specific changes), the intent log is cleared, and the service is started.

[0072] Reference Figure 7 , Figure 7 The diagram shown is a schematic of the intent log and main database storage design provided by the present invention. Figure 7 As shown, it includes: Intent Log: Transaction intent and resource usage log, independent of the main data file, is an intent log file used for appending writes.

[0073] Three scenarios: 1) Lightweight transaction commit: The feature is that it no longer persists all the pages involved in the transaction, but only persists the transaction ID and the page number that was modified, and generates an intent log.

[0074] 2) Periodic asynchronous checkpoint flushing: Periodically perform checkpoint operations to update the status of the intent log.

[0075] 3) Perform crash recovery operations: Use intent logs to restore the complete database.

[0076] The characteristics of intent logs include: 1. It only records the transaction ID and its page ID list. Compared to recording the transaction operation content, the cost is very small. Most of the content involved in the transaction can be persisted in one page, and the performance of each persistence is relatively high.

[0077] 2. In some application scenarios, if a separate intent log file is not desired, the intent log can be recorded in a specific page of the database file. For example... Figure 7 As shown, the intent log pages are set to occupy 10 pages, using reserved page numbers from 2 to 11.

[0078] The following reference Figure 8 To describe an electronic device 800 according to this embodiment of the present invention. Figure 8 The electronic device 800 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0079] like Figure 8 As shown, the electronic device 800 is manifested in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including storage unit 820 and processing unit 810).

[0080] The storage unit stores program code that can be executed by the processing unit 810, causing the processing unit 810 to perform the steps described in the "Embodiment Methods" section of this specification according to various exemplary embodiments of the present invention.

[0081] Storage unit 820 may include readable media in the form of volatile storage units, such as random access memory (RAM) 821 and / or cache memory 822, and may further include read-only memory (ROM) 823.

[0082] The storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0083] Bus 830 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0084] Electronic device 800 can also communicate with one or more external devices (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 800, and / or any device that enables electronic device 800 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 850. Furthermore, electronic device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 860. As shown, network adapter 860 communicates with other modules of electronic device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0085] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0086] According to the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the present invention can also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the present invention described in the "Exemplary Methods" section above.

[0087] refer to Figure 9 As shown, a program product 900 for implementing the above-described method according to an embodiment of the present invention is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0088] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0089] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0090] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0091] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0092] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0093] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

[0094] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

Claims

1. An embedded database transaction processing method, characterized in that, include: Based on the database write transaction request, a new data page is allocated for the transaction in the database and the write is completed on the data page; Record the identifier of the transaction and the list of allocated data page resources to generate an intent log; In response to a preset trigger condition, the transactions on the data page are batch-written to disk and then to the database. If an anomaly occurs during the disk write process, the data page resource list recorded in the intent log will be used to reclaim page resources in order to restore the complete database.

2. The method according to claim 1, characterized in that, If an anomaly occurs during the disk write process, the page resources will be reclaimed using the data page resource list recorded in the intent log, including: In response to a crash and restart during disk write, the database state is used as a baseline and the list of data page resources is extracted based on the records in the intent log. The page resources corresponding to the data page resource list are recycled to the database's idle page list to restore the database to a consistent, ready-to-use state.

3. The method according to claim 1, characterized in that, The step of allocating a new data page for the transaction in the database and completing the write operation on the data page based on the database write transaction request includes: If a database write transaction commit request is received, then in the memory-mapped area of ​​the database file, a new data page is allocated for the transaction using the copy-on-write mechanism; Write the data changes corresponding to the transaction to the data page.

4. The method according to claim 1, characterized in that, Before extracting the data page resource list based on the intent log records read, the process includes: Determine if the intent log file exists and contains log records; If log records exist and are included, discard all data changes made by transactions in the log records.

5. The method according to claim 1, characterized in that, After recording the identifier of the transaction and the list of allocated data page resources to generate the intent log, the process includes: Perform a disk synchronization operation on the intent log to persist the intent log, and return a successful transaction commit to the client without synchronizing the database files.

6. The method according to claim 1, characterized in that, The step of batch writing transactions on the data page to the database in response to a preset trigger condition includes: Determine whether the preset trigger conditions are met. The preset trigger conditions include: whether the set timer has timed out, whether the intent log threshold meets the set conditions, or whether a processing instruction has been received. If the preset triggering conditions are met, the transactions written on the new data page will be batch synchronized to the database, thus making the current state of the memory-mapped area of ​​the database file persistent.

7. The method according to claim 6, characterized in that, Also includes: Once it is confirmed that the transaction has been successfully synchronized to the database, truncate or clear the intent log file.

8. An embedded database transaction processing device, characterized in that, include: An allocation module is used to allocate a new data page for the transaction in the database and complete the writing on the data page based on the database write transaction request. A generation module is used to record the identifier of the transaction and the list of allocated data page resources to generate an intent log; An execution module is used to, in response to a preset trigger condition, batch write transactions on the data page to disk and then to the database. The processing module is used to reclaim page resources using the data page resource list recorded in the intent log if an abnormality occurs during the disk write process, so as to restore the complete database.

9. An electronic device, characterized in that, The electronic device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores computer program instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 7.

Images