Database write-ahead log processing method, system recovery method and related equipment

Abstract

The embodiments of the present application provide a method for processing a database pre-written log, a system recovery method, and related equipment, and belong to the field of database management. The method includes: obtaining the log global sequence number and data page modification information of the current transaction; if the data page modification information indicates that the current transaction involves cross-data page modification, waiting for the pre-written log corresponding to the current transaction to be written to the preset disk by the log write disk unit and all pre-written logs with log global sequence numbers less than the current log global sequence number to be written to the disk by the corresponding log write disk unit, and then committing the current transaction; if the data page modification information indicates that the current transaction does not involve cross-data page modification, waiting for the pre-written log corresponding to the current transaction to be written to the disk by the corresponding log write disk unit, and then committing the current transaction. The present application prevents log writes from becoming a single point bottleneck of the system, and transaction execution units that modify different data independently commit transactions without waiting for unrelated log write disk units to write logs to disk.

Description

Technical Field

[0001] The present application relates to the technical field of database management, and in particular to a database write-ahead log processing method, a system recovery method and related equipment. Background Art

[0002] In a database management system, data updates occur in data pages in the memory buffer. For performance purposes, data pages in the buffer will not be required to be written to disk when the transaction that modifies it is committed. Instead, a write-ahead log is used to ensure the atomicity and durability of data operation transactions in the database.

[0003] In the related technology, write-ahead logs are used to complete data writing. However, in common database management systems, write-ahead logs are written to disk serially, and transaction submission must rely on logs being written to disk, making the performance of log writing a bottleneck for the entire database performance. Therefore, how to prevent log writing from becoming a bottleneck for database management systems has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a database pre-write log processing method, system recovery method and related equipment, aiming to realize parallel log writing, so that log disk writing does not become a single point bottleneck of the system, and transaction execution units that modify different data submit transactions independently without waiting for unrelated log disk writing units to write logs to disk, thereby improving the efficiency of log writing.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for processing a write-ahead log of a database, which is applied to a process model of multiple transaction execution units and multiple log writing units, and the multiple transaction execution units execute transactions in parallel, and the log writing units write logs in parallel, and the method includes:

[0006] Obtaining log write disk associated data of the current transaction; wherein the log write disk associated data includes the current log global sequence number and data page modification information, and the current log global sequence number is the log global sequence number of the current transaction maintained by the transaction execution unit;

[0007] If the data page modification information indicates that the current transaction involves cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be written to the preset disk by the log writing unit, and for all the write-ahead logs whose log global sequence numbers are smaller than the current log global sequence number to be written to the disk by the corresponding log writing unit, and then commit the current transaction;

[0008] If the data page modification information indicates that the current transaction does not involve cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be written to the disk by the corresponding log writing unit, and then commit the current transaction.

[0009] In some embodiments, one of the transaction execution units or one of the log writing units is driven by a thread or a coroutine, each of the transaction execution units is provided with a write-ahead log buffer, each of the log writing units processes the write-ahead log buffers of one or more transaction execution units in parallel, each of the log writing units has its own independent write-ahead log disk storage path, and before obtaining the log writing disk associated data of the current transaction, the method further includes:

[0010] When the current transaction starts, obtaining log global sequence data of the current transaction;

[0011] When processing a data page according to a preset operation, the log global sequence data of the current transaction and data page related parameters of the data page are maintained, and a write-ahead log is generated and written into a write-ahead log buffer; wherein the data page related parameters are used to determine the data page modification information.

[0012] In some embodiments, the log global sequence data includes: a transaction number, a log global sequence number, and a minimum sequence number. When the current transaction starts, obtaining the log global sequence data of the current transaction includes:

[0013] When the current transaction starts, obtaining the transaction number of the current transaction;

[0014] Obtain the log global sequence number of the current transaction;

[0015] Obtain the minimum value of the largest log global sequence number of the current transaction to obtain the minimum sequence number.

[0016] In some embodiments, the preset operation includes: access, single non-cross-page, single cross-page modification, and when processing the data page according to the preset operation, maintaining the log global sequence data of the current transaction and the data page related parameters of the data page, and generating a write-ahead log to write into the write-ahead log buffer, including:

[0017] When accessing a data page, setting the current log global sequence number of the current transaction to the maximum value of the current log global sequence number and the log global sequence number of the data page;

[0018] When a data page is modified in a single non-cross-page manner, the log global sequence number of the data page remains unchanged, and the modification times of the data page are incremented;

[0019] When a data page is modified across pages in a single time, the current global log sequence number of the current transaction is incremented, the global log sequence number of the data page is set to the current global log sequence number of the current transaction, and the number of modifications of the data page is incremented, and the transaction number of the last modification transaction execution unit of the data page is changed to the transaction number of the current transaction execution unit;

[0020] Generate a write-ahead log and write it into the write-ahead log buffer.

[0021] In some embodiments, after the generated write-ahead log is written into the write-ahead log buffer, the method further includes:

[0022] The log global sequence number for generating the write-ahead log record is set to the maximum value of the log global sequence number of the current transaction when modifying the data page and the log global sequence number of the data page.

[0023] In some embodiments, if the data page modification information indicates that the current transaction involves cross-data page modification, waiting for the write-ahead log corresponding to the current transaction to be written to a preset disk by the log writing unit, and for all write-ahead logs whose log global sequence numbers are smaller than the current log global sequence number to be written to the disk by the corresponding log writing unit, and then committing the current transaction, includes:

[0024] If the data page modification information indicates that the current transaction involves cross-data page modification, obtain the maximum value of the log global sequence number of the written logs in each of the log writing units to obtain the log global sequence number of the log writing unit; wherein the written log is a pre-written log that has been written to the disk by the log writing unit;

[0025] Monitoring the log writing unit where the current transaction to be committed is located to write all pre-written logs corresponding to the current transaction to the disk;

[0026] Monitor that the log global sequence numbers of all the log writing units are greater than or equal to the current log global sequence number of the current transaction, and commit the current transaction.

[0027] In some embodiments, monitoring that the log global sequence numbers of all the log writing units are greater than or equal to the current log global sequence number of the current transaction and submitting the current transaction includes:

[0028] Obtain remote log disk dependency mark information; wherein, at the beginning of the current transaction, obtain the minimum value of the log global sequence number of all log writing units and record it as the visible disk log global sequence number of the current transaction, and set the remote log disk dependency mark information to false; when accessing a data page, if the log global sequence number of the accessed data page is greater than the visible disk log global sequence number of the current transaction, and the transaction number of the last modified transaction execution unit of the data page is different from the transaction number of the current transaction execution unit, then set the remote log disk dependency mark information to true;

[0029] If the remote log disk dependency mark information is true, monitor the log global sequence numbers of all the log writing units, wait until the log global sequence numbers of all the log writing units are greater than or equal to the current log global sequence number of the current transaction, and monitor the log writing unit where the current transaction to be committed is located to write all the pre-written logs of the previous transaction to the disk, and commit the current transaction;

[0030] If the remote log disk dependency mark information is false, the log disk writing unit where the current transaction to be committed is monitored writes all pre-written logs of the current transaction to the disk, and then commits the current transaction.

[0031] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a system recovery method, wherein a write-ahead log is obtained by the write-ahead log processing method of the database as described in the first aspect, and the method comprises:

[0032] Obtain a regularly set log checkpoint, and use the log checkpoint as a starting point;

[0033] When the system is restarted, starting from the starting point, a redo operation is performed according to the redo log, a rollback operation is performed according to the rollback log, and a new rollback operation is determined according to the transaction log to restore the system; wherein the redo log is a write-ahead log of the redo category when modifying data, the rollback log is a write-ahead log that records the rollback category during the rollback process of the current transaction, and the transaction log is a write-ahead log of the transaction category when the current transaction is started, committed, and rolled back.

[0034] In some embodiments, before obtaining a periodically set log checkpoint and using the log checkpoint as a starting point, the method further includes:

[0035] Setting the log checkpoint specifically includes:

[0036] Partitioning the system data buffer to obtain data buffer partitions, periodically polling and selecting a data buffer partition, writing all data pages therein to a disk, and using the minimum value of the global log sequence number of the data page as the global log sequence number of the data buffer partition;

[0037] Obtaining the minimum value of the global log sequence numbers of all data buffer partitions as the global log sequence number of the system data buffer;

[0038] Get the minimum value of the global log sequence number of all active transactions as the minimum active transaction global log sequence number;

[0039] The minimum value of the global log sequence number of the system data buffer and the minimum active transaction global log sequence number is set as the log checkpoint.

[0040] In some embodiments, before the system is restarted, starting from the starting point, performing a redo operation according to the redo log, performing a rollback operation according to the rollback log, and determining a new rollback operation according to the transaction log to restore the system, the method further includes:

[0041] Starting scanning from the starting point, a plurality of log writing units generate each log file;

[0042] If a target transaction starts to operate the log file during scanning the log file, the target transaction is added to the active transaction list, and when the target transaction commits and rolls back the write-ahead log, the target transaction is removed from the active transaction list;

[0043] If the write-ahead log encountered during scanning of the log file is the redo log and the rollback log, the write-ahead logs are grouped according to the modified data page sequence numbers, the write-ahead logs in the same group are sorted according to the log global sequence number and the number of modification times of the data page, and the data log information of the transaction corresponding to the write-ahead log in the active transaction list is updated;

[0044] The active transaction list after scanning all write-ahead logs is obtained, and all target transactions and the data log information corresponding to the target transactions are added to a rollback list.

[0045] To achieve the above-mentioned purpose, a third aspect of an embodiment of the present application proposes a write-ahead log processing device for a database, which is applied to a process model of multiple transaction execution units and multiple log writing units, and the multiple transaction execution units execute transactions in parallel, and the log writing units write logs in parallel, and the device includes:

[0046] A data acquisition module, used to acquire log write disk associated data of the current transaction; wherein the log write disk associated data includes the current log global sequence number and data page modification information, and the current log global sequence number is the log global sequence number of the current transaction maintained by the transaction execution unit;

[0047] A first transaction processing module is configured to, if the data page modification information indicates that the current transaction involves cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be written to a preset disk by the log writing unit, and for all write-ahead logs whose log global sequence numbers are smaller than the current log global sequence number to be written to the disk by the corresponding log writing unit, and then commit the current transaction;

[0048] The second transaction processing module is used to wait for the write-ahead log corresponding to the current transaction to be written to the disk by the corresponding log writing unit if the data page modification information indicates that the current transaction does not involve cross-data page modification, and then commit the current transaction.

[0049] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the write-ahead log processing method of the database described in the first aspect above, or the system recovery method described in the second aspect is implemented.

[0050] The write-ahead log processing method, system recovery method and related equipment of the database proposed in the present application first determine whether the current transaction involves cross-data page modification. If it involves cross-data page modification, it indicates that it is necessary to wait for the write-ahead log corresponding to the current transaction to be written to disk, and the write-ahead log with a log global sequence number less than the transaction sequence number to be written to disk before the current transaction is committed. If the current transaction does not involve cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be committed before submitting the current transaction. Therefore, for the two situations of whether the data page is modified across data pages, the write-ahead log is submitted after the global sequence number of the log is combined to submit the current transaction. At the same time, the transaction submission is executed in parallel by multiple transaction execution units, which not only improves the processing efficiency of the write-ahead log and transactions, but also allows the write-ahead log and transactions to be processed in an orderly manner. Therefore, the write-ahead log processing method of the database proposed in the present application can not only improve the transaction processing efficiency and achieve higher parallel write disk functions, but also ensure that different transaction processing on the same data page has waiting dependencies without affecting data confusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a system framework diagram of a write-ahead log processing method for a database provided in an embodiment of the present application;

[0052] Figure 2 It is a flowchart of a write-ahead log processing method for a database provided in an embodiment of the present application;

[0053] Figure 3 is a flowchart of a write-ahead log processing method for a database provided by another embodiment of the present application;

[0054] Figure 4 yes Figure 3 Flow chart of step S301 in FIG.

[0055] Figure 5 yes Figure 3 Flow chart of step S302 in FIG.

[0056] Figure 6 yes Figure 2 Flow chart of step S202 in FIG.

[0057] Figure 7 yes Figure 6 Flow chart of step S603 in FIG.

[0058] Figure 8 is a flowchart of a system recovery method provided by an embodiment of the present application;

[0059] Fig. 9 is a flowchart of a system recovery method provided by another embodiment of the present application;

[0060] Fig.10 is a flowchart of a system recovery method provided by another embodiment of the present application;

[0061] Fig.11 It is a structural schematic diagram of a write-ahead log processing device for a database provided in an embodiment of the present application;

[0062] Fig.12 It is a structural diagram of the system recovery device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] First, some nouns involved in this application are analyzed:

[0067] Write-Ahead Logging: is a standard way to implement transaction logging. The central idea of ​​write-ahead logging is that modifications to data files (which are the carriers of tables and indexes) must only occur after these modifications have been logged, that is, after the log records describing these changes have been flushed to permanent storage. If we follow this process, then we do not need to flush the data pages to disk every time a transaction is committed, because we know that in the event of a crash, we can use the log to recover the database, and any records not attached to the data pages will be redone from the log records first (this is called rolling forward recovery, also known as REDO).

[0068] Database Management System (DBMS): It is a large-scale software for manipulating and managing databases. It is used to establish, use and maintain databases. The DBMS manages and controls the database in a unified way to ensure the security and integrity of the database. Users access the data in the database through the DBMS, and database administrators also maintain the database through the DBMS. It can support multiple applications and users to establish, modify and query the database in different ways at the same time or at different times.

[0069] GSN (Global Sequence Number): used to maintain the global sequence relationship between different write-ahead logs.

[0070] PLSN (page log sequence number): A counter maintained by each page, which is incremented by 1 when the page is modified.

[0071] Thread: It is the smallest unit that the operating system can schedule operations. Threads are contained in processes and are the actual operating units in processes. A thread refers to a single sequential control flow in a process. Multiple threads can run concurrently in a process, and each thread executes different tasks in parallel.

[0072] Checkpoint: It is a database event. Its fundamental purpose is to reduce the time of crash recovery. When modifying data, you need to read the data into the memory (Buffer Cache) first. While modifying the data, Oracle will record the redo information (Redo) for recovery. Because of the existence of redo information, Oracle does not need to write the changed data back to the disk immediately when submitting (immediate writing will be very inefficient). The existence of redo (Redo) is also to restore the data after the database crashes.

[0073] In a database management system, data updates occur in the data pages of the memory buffer. For performance purposes, the buffered data pages are not required to be written to disk when the transaction that modifies them is committed. Instead, a write-ahead log is used to ensure the atomicity and durability of data operation transactions in the database, that is, all modification logs must be written to the log file before the transaction is committed. During the recovery phase, the database can restore the latest version of data based on the older version of data stored on the disk and replay the write-ahead log.

[0074] In common database management systems, such as PostgreSQL, even if writing table data to disk can be done in parallel, the write-ahead log is written to disk in serial mode, and transaction commits must rely on the log being written to disk, so the log writing performance will become the performance bottleneck of the entire database system. The classic serial write-to-disk method is also fully described in ARIES. In order to solve the performance bottleneck of serial write-ahead logs, openGauss and MySQL have implemented their own parallel write log buffers to increase the parallelism of log writing, but in fact, the way to write the write-ahead log to the file is still serial.

[0075] To solve the problem of serial log writing, Distributed Logging proposes a distributed log solution, where transactions executed in parallel write their own local log files. At the same time, to ensure that logs can be redone in the order of transaction execution during recovery, these logs spanning log files record the GSN maintained by the system and are sorted in GSN order during recovery.

[0076] However, it is found that transactions that modify different data are logically independent even if the GSN size is different, and the submission does not require the reliance on logs being written to disk. Therefore, the RFA solution is proposed. When a transaction is submitted, it determines the data pages that it has modified. If all transactions that modified these pages before the transaction itself have been submitted and the corresponding logs have been written to disk, the transaction can be submitted without waiting for the pre-write log processing of other databases with smaller global sequence numbers to be completed.

[0077] The ARIES solution is the most basic serial solution and does not have performance optimization for parallel disk writing. To solve the performance bottleneck of serial log writing, openGauss and MySQL have implemented their own parallel log buffer write methods to increase the parallelism of log writing. However, the log buffer is still written to the log file serially through a background thread, and the log file writing still has the bottleneck of serial writing.

[0078] Distributed Logging is the most basic parallel solution, but parallel transaction submissions still need to wait based on the global sequence number, that is, the transaction of a partition must wait for all other smaller (smaller GSN) databases to successfully process the pre-written logs. Therefore, although the logs are written to the disk in parallel, the latency of transaction submission is still very high.

[0079] Although the RFA solution implements a parallel write-ahead log solution and solves the problem of high transaction submission latency in Distributed Logging, the global sequence numbers of the logs are bound to be large and small, and the transactions of the log GSN still cannot avoid the dependence on the log write disk of transactions with small log global sequence numbers.

[0080] Based on this, the embodiment of the present application provides a database write-ahead log processing method, system recovery method and related equipment, which first determines whether the current transaction involves cross-data page modification. If it involves cross-data page modification, it indicates that it is necessary to wait for the write-ahead log corresponding to the current transaction to be written to disk, and the write-ahead log with a log global sequence number less than the transaction sequence number to be written to disk before submitting the current transaction. If the current transaction does not involve cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be submitted, and then submit the current transaction. Therefore, for the two situations of whether the data page is modified across data pages, the current transaction is submitted after the write-ahead log submission is completed in combination with the log global sequence number. At the same time, the transaction submission is executed in parallel by multiple transaction execution units, which not only improves the processing efficiency of the write-ahead log and transactions, but also allows the write-ahead log and transactions to be processed in an orderly manner. Therefore, the write-ahead log processing method for the database proposed in the present application can not only improve the transaction processing efficiency and achieve higher parallel write disk functions, but also ensure that different transaction processing on the same data page has waiting dependencies, without affecting data confusion.

[0081] The write-ahead log processing method, system recovery method and related devices of the database provided in the embodiments of the present application are specifically described through the following embodiments. First, the write-ahead log processing method of the database in the embodiments of the present application is described.

[0082] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0083] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0084] The pre-written log processing method of the database provided in the embodiment of the present application relates to the field of database management technology. The pre-written log processing method of the database provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or it can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms and other basic cloud computing services. Cloud servers; software can be applications that implement pre-written log processing methods for databases, etc., but are not limited to the above forms.

[0085] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0086] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0087] Please refer to Figure 1 , Figure 1 is a schematic diagram of a model of a write-ahead log processing method for a database provided in an embodiment of the present application, through Figure 1 It can be seen that the write-ahead log processing method of the database of the embodiment of the present application needs to involve a process model, which includes an executor, a memory, a write-ahead log writing disk and a disk. Multiple transaction execution units are set in the executor, and each transaction execution unit runs transactions in parallel. A transaction execution unit can be driven by a thread or a coroutine, and each transaction execution unit has an exclusive write-ahead log buffer. Each transaction execution unit will first write the write-ahead log to the memory, but writing to the memory area is not safe. The memory will regularly clean up the write-ahead log, so it is necessary to write the write-ahead log to the disk. A log write disk unit is provided in the write-ahead log writing disk, and the log write disk unit is connected to the memory area, and docks the write-ahead logs of multiple memory areas, and stores the write-ahead logs from the memory to the disk. It should be noted that the security of storing the write-ahead log in the disk is relatively strong and is not easy to be cleaned up.

[0088] Specifically, during the transaction execution process, there is no competition between constructing write-ahead log records and writing buffers. Since the log writing parallelism does not have to be consistent with the transaction parallelism, the present invention supports grouping transaction execution units, and each log writing unit can process the write-ahead log buffers of multiple parallel transaction execution units. Each log writing unit has its own independent write-ahead log disk storage path, and the write behaviors of the log writing unit are independent and parallel, and the files of the write-ahead log on the disk are also independent of each other.

[0089] It can be seen that the successful submission of a transaction by a transaction execution unit depends on the successful processing of the database's write-ahead log generated by itself, and on the completion of the disk flushing of the write-ahead log by other parallel transaction execution units. Therefore, when a transaction execution unit executes a transaction, it finds and obtains the minimum dependency set of other parallel transactions that must be waited for, that is, maximizes the disk writing performance of the parallel write-ahead log.

[0090] Figure 2It is an optional flow chart of a database write-ahead log processing method provided by an embodiment of the present application. The database write-ahead log processing method is applied to a process model of multiple transaction execution units and multiple log writing units, and multiple transaction execution units execute transactions in parallel, and log writing units write logs in parallel. Figure 2 The method may include but is not limited to steps S201 to S203.

[0091] Step S201, obtaining log write disk associated data of the current transaction; wherein the log write disk associated data includes the current log global sequence number and data page modification information, and the current log global sequence number is the log global sequence number of the current transaction maintained by the transaction execution unit;

[0092] Step S202: If the data page modification information indicates that the current transaction involves cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be written to the preset disk by the log writing unit, and for all the write-ahead logs whose log global sequence numbers are smaller than the current log global sequence number to be written to the disk by the corresponding log writing unit, and then commit the current transaction;

[0093] Step S203: if the data page modification information indicates that the current transaction does not involve cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be written to the disk by the corresponding log writing unit, and then commit the current transaction.

[0094] Steps S201 to S203 shown in the embodiment of the present application first determine whether the current transaction involves cross-data page modification. If it involves cross-data page modification, it indicates that it is necessary to wait for the write-ahead log corresponding to the current transaction to be written to disk, and the write-ahead log with a log global sequence number less than the transaction sequence number to be written to disk before the current transaction is committed. If the current transaction does not involve cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be committed before committing the current transaction. Therefore, for the two situations of whether the data page is modified across data pages, the current transaction is submitted after the write-ahead log is submitted in combination with the log global sequence number. At the same time, the transaction submission is executed in parallel by multiple transaction execution units, which not only improves the processing efficiency of the write-ahead log and transactions, but also allows the write-ahead log and transactions to be processed in an orderly manner. Therefore, the write-ahead log processing method for the database proposed in the present application can not only improve the transaction processing efficiency and achieve higher parallel disk writing functions, but also ensure that different transaction processing on the same data page has waiting dependencies without affecting data confusion.

[0095] In step S201 of some embodiments, the log global sequence number is defined as GSN, which is used to maintain the global order relationship between different write-ahead logs. It should be noted that the page level is set to avoid remote refresh dependency, and the update logic of the log global sequence number is modified. The global sequence number will only be updated when the updated data of the data page causes a cross-data page change, and the cross-data page change can be data page creation / splitting / merging. It should be noted that the log global sequence number is set so that it will not be frequently modified, thereby avoiding the problem of two independent transactions having the same log global sequence number due to concurrent modification of different data on the same data page.

[0096] As disclosed above, the current log global sequence number of the current transaction is obtained, and the current log global sequence number of the current transaction is defined as txn GSN, which is also the maximum GSN of the page modified by the current transaction.

[0097] In some embodiments, Figure 3 As shown, before step S201, the database write-ahead log processing method may also include but is not limited to steps S301 to S302:

[0098] Step S301, when the current transaction starts, obtaining the log global sequence data of the current transaction;

[0099] Step S302, when processing the data page according to the preset operation, maintain the log global sequence data of the current transaction and the data page related parameters of the data page, and generate a write-ahead log and write it into the write-ahead log buffer; wherein the data page related parameters are used to determine the data page modification information.

[0100] In step S301 of some embodiments, the current transaction is a transaction currently executed by one of the transaction execution units. The log global sequence data includes: transaction number, log global sequence number, and minimum sequence number. The transaction number is also defined as the transaction number of the most recently updated page transaction. The log global sequence number is defined as GSN, and the minimum sequence number is the minimum value of the maximum log global sequence number.

[0101] Please refer to Figure 4 In some embodiments, step S301 may include but is not limited to steps S401 to S403:

[0102] Step S401, when the current transaction starts, obtain the transaction number of the current transaction;

[0103] Step S402, obtaining the log global sequence number of the current transaction;

[0104] Step S403, obtaining the minimum value of the maximum log global sequence number of the current transaction to obtain the minimum sequence number.

[0105] It should be noted that if the current transaction starts, obtain the minimum value of the maximum log global sequence number and the log global sequence number of the current transaction flushed by each log writing thread, and the log global sequence number is the maximum log global sequence number of the system at the start time of the current transaction. After the log global sequence number and the minimum sequence number are obtained, initialize needsRemoteFlush=false. Specifically, needsRemoteFlush=false is equivalent to setting the commit of the current transaction without waiting for all pre-written logs with log global sequence numbers smaller than the current log global sequence number of the current transaction to be written to the disk, and transaction logs with the same log global sequence number can be written in parallel, which improves the efficiency of transaction commit and allows transaction processing to proceed in an orderly manner.

[0106] In step S302 of some embodiments, when a preset operation is performed on a data page, the transaction execution unit maintains the log global sequence data of the current transaction and the data page related parameters of the data page, and the log execution threads corresponding to each transaction execution unit respectively maintain their own log global sequence number and data page related parameters to achieve separate data maintenance and prevent confusion in the log global sequence number and data page related parameters.

[0107] Specifically, the preset operations include: access, single non-cross-page, and single cross-page modification. Single non-cross-page means that completing the current transaction on the same data page will cause data updates that will not cause cross-data page changes. Single cross-page modification means that the data updates caused by executing the current transaction will cause cross-data page changes. Data page related parameters include: the number of modifications to the data page and the log global sequence number. The number of modifications to the data page is defined as PLSN, and each data page maintains its own calculator, which automatically increases by 1 when the data page is modified, so as to determine the number of times the data page is modified by the number of modifications. The log global sequence number of the data page is the log global sequence number used when modifying the data page. It should be noted that the order of different write-ahead logs on the same data page can be determined by the number of modifications to the data page and the log global sequence number.

[0108] Please refer to Figure 5 In some embodiments, step S302 may include but is not limited to steps S501 to S504:

[0109] Step S501, when accessing a data page, setting the current log global sequence number of the current transaction to the maximum value between the current log global sequence number and the log global sequence number of the data page;

[0110] Step S502, when a data page is modified in a single non-cross-page manner, the log global sequence number of the data page remains unchanged, and the modification times of the data page are incremented;

[0111] Step S503, when a data page is modified across pages in a single time, the current log global sequence number of the current transaction is incremented, the log global sequence number of the data page is set to the current log global sequence number of the current transaction, and the modification times of the data page are incremented, and the transaction number of the last modification transaction execution unit of the data page is changed to the transaction number of the current transaction execution unit;

[0112] Step S504: Generate a write-ahead log and write it into a write-ahead log buffer.

[0113] In step S501 of some embodiments, when a data page is accessed, the data page is locked, and then the current log global sequence number of the current transaction is obtained, and then the log global sequence number of the data page is obtained, and the maximum value between the current log global sequence number of the current transaction and the log global sequence number of the data page is used as the current log global sequence number of the current transaction. Specifically, the current log global sequence number of the current transaction is defined as txn GSN, and the log global sequence number of the data page is defined as page GSN, so txn GSN=max{txn GSN,page GSN} is set, and the largest log global sequence number between the two is set as the log global sequence number of the current transaction to ensure orderly execution of transactions.

[0114] In steps S502 to S503 of some embodiments, by determining whether the preset operation is a single non-cross-page modification of the data page or a single cross-page modification of the data page, that is, determining whether the updated data causes a cross-page migration, the log global sequence number of the current transaction or data page is set according to different situations. Specifically, if the preset operation is a single non-cross-page modification of the data page, that is, the updated data does not cause a cross-page migration, the log global sequence number of the data page remains unchanged, and the number of modifications of the data page increases, and the number of modifications of the data page is defined to increase as PLSN++. If the preset operation is to modify the data page across pages in a single time, that is, update the data resulting in a cross-data page change, the current log global sequence number of the current transaction is incremented, the log global sequence number of the data page is set to the current log global sequence number of the current transaction, and the number of modifications to the data page is incremented, and the transaction number of the last modified transaction execution unit of the data page is changed to the transaction number of the current transaction execution unit. The process is defined as: set txn GSN++ and update txn GSN=max{txn GSN, page GSN} at the same time, and set pagelastWrittenXid = current xid. It should be noted that txn GSN++ represents the increment of the current log global sequence number of the current transaction, txn GSN = max{txn GSN, page GSN} represents the maximum value of the updated log global sequence number of the current transaction and the data page as the current log global sequence number of the current transaction, and page lastWrittenXid=current xid represents the change of the transaction number of the last modified transaction execution unit of the data page to the transaction number of the current transaction execution unit.

[0115] It should be noted that in order to determine whether there is a cross-data page transition, the global log sequence number of the data page and the current transaction is adjusted to set the log global sequence number to be monotonically increasing but not unique (when there is no cross-data page transition, a large number of data pages, transactions, and write-ahead logs will have the same log global sequence number), that is, to determine when the log global sequence number of the data page and transaction changes.

[0116] In steps S501 to S504 shown in this embodiment, the log global sequence number of the data page and the log global sequence number of the current transaction are set according to different situations, so that the processing of maintaining the write-ahead log according to the log global sequence number is more orderly.

[0117] In some embodiments, after step S504, the database write-ahead log processing method may further include: generating a log global sequence number of a write-ahead log record and setting it to the maximum value of the log global sequence number of the current transaction and the log global sequence number of the data page when modifying the data page.

[0118] It should be noted that after the write-ahead log is written into the write-ahead log buffer, a log global sequence number is set for the write-ahead log, and the log global sequence number of the write-ahead log is defined as the maximum value of the log global sequence number of the current transaction and the log global sequence number of the data page when modifying the data page, so that the write-ahead log can be written into the preset buffer in an orderly manner according to the log global sequence number of the write-ahead log.

[0119] In step S202 of some embodiments, before the current transaction is committed, it is necessary to determine whether the current transaction is a cross-data page transition or not. Therefore, it is necessary to perform a cross-data page detection on the current transaction to obtain data page modification information. It should be noted that the change of data page related parameters can determine the data page modification information.

[0120] Please refer to Figure 6 In some embodiments, step S202 may include but is not limited to steps S601 to S603:

[0121] Step S601: If the data page modification information indicates that the current transaction involves cross-data page modification, obtain the maximum value of the log global sequence number of the written logs in each log writing unit to obtain the log global sequence number of the log writing unit; wherein the written log is a pre-written log that has been written to the disk by the log writing unit;

[0122] Step S602, monitoring the log writing unit of the current transaction to be committed to write all the pre-written logs corresponding to the current transaction to the disk;

[0123] Step S603: monitor that the log global sequence numbers of all log writing units are greater than or equal to the current log global sequence number of the current transaction, and commit the current transaction.

[0124] In step S601 of some embodiments, if the data page modification information indicates that the current transaction involves cross-data page modification, it indicates that the updated data causes cross-data page migration. Therefore, the maximum value of the log global sequence number flushed by each log writing thread is obtained, that is, the maximum value of the log global sequence number of the written log is determined.

[0125] In step S602 of some embodiments, by monitoring the log write disk unit where the current transaction to be committed is located, it is detected that the log write disk unit is writing to the disk to determine whether the pre-written log corresponding to the current transaction has been written to the disk, so as to determine that the next judgment can be made after the pre-written log is written to the disk.

[0126] In step S603 of some embodiments, if the write-ahead log corresponding to the current transaction has completed writing to disk, further judgment is required, that is, to judge that the current transaction submission needs to wait for all the write-ahead logs with global sequence numbers less than the current transaction sequence number to be written to disk. On the other hand, by judging that the log global sequence number of the log writing unit is better than the current log global sequence number of the current transaction, that is, determining that the write-ahead log with a log global sequence number less than the current log global sequence number of the current transaction has completed writing, the current transaction is committed.

[0127] In steps S601 to S603 shown in this embodiment, when it is determined that the current transaction involves cross-data page modification, it is first determined that the write-ahead log corresponding to the current transaction is written to disk, and then the current transaction is committed after waiting for the write-ahead log with a smaller log global sequence number to be written to disk, so that the transaction write-ahead log disk operation can be completed in an orderly manner.

[0128] In some embodiments, please refer to Figure 7 Step S603 may include but is not limited to steps S701 to S703:

[0129] Step S701, obtaining remote log disk dependency mark information; wherein, at the beginning of the current transaction, obtaining the minimum value of the log global sequence number of all log writing units and recording it as the visible disk log global sequence number of the current transaction, and setting the remote log disk dependency mark information to false; when accessing a data page, if the log global sequence number of the accessed data page is greater than the visible disk log global sequence number of the current transaction, and the transaction number of the last modified transaction execution unit of the data page is different from the transaction number of the current transaction execution unit, then setting the remote log disk dependency mark information to true;

[0130] Step S702: If the remote log write dependency mark information is true, monitor the log global sequence numbers of all log write units, wait until the log global sequence numbers of all log write units are greater than or equal to the current log global sequence number of the current transaction, and monitor the log write unit where the current transaction to be committed is located to write all pre-written logs of the current transaction to disk, and commit the current transaction;

[0131] Step S703: If the remote log disk dependency mark information is false, monitor the log disk writing unit where the current transaction to be committed is located to write all pre-written logs of the current transaction to the disk, and then commit the current transaction.

[0132] It should be noted that the remote log flushing dependency mark information is defined as needsRemoteFlush. If needsRemoteFlush==false, it indicates that the remote log flushing dependency mark information is false. If needsRemoteFlush=true, it indicates that the remote log flushing dependency mark information is true.

[0133] Specifically, if the current transaction is only internally committed and no log is written to disk, the current transaction identification number of the current transaction needs to be added to the write-ahead log transaction table, which is rfaChecksAtPrecommit. rfaChecksAtPrecommit is used to determine whether all dependent transactions have been committed when the transaction is committed. Therefore, it is necessary to traverse and determine whether all the write-ahead logs in rfaChecksAtPrecommit have been written to disk. If they have not been written to disk, that is, needsRemoteFlush=true is determined, then the log global sequence numbers of all log write disk units are monitored to be greater than or equal to the current log global sequence number of the current transaction, and the current transaction is submitted after the write-ahead log of the current transaction is written to disk.

[0134] Traverse and determine that all the pre-written logs in rfaChecksAtPrecommit are written to disk, and determine that the remote log disk dependency mark information is false. The transaction number of the current transaction needs to be added to precommittedQueueRfa, indicating that the current transaction can be committed only after waiting for its own pre-written log to be written to disk.

[0135] Therefore, based on the remote log disk dependency mark information, it is determined whether it is necessary to wait for the pre-written log with a log global sequence number greater than or equal to the current transaction's current log global sequence number to be written to disk, or whether it is necessary to wait for the current transaction's own pre-written log to be written to disk before committing the current transaction, to ensure that the pre-written log is written to disk in an orderly manner and transaction processing is also completed in an orderly manner.

[0136] In addition, please refer to Figure 8 This embodiment also discloses a system recovery method, which obtains a write-ahead log by the above-mentioned write-ahead log processing method of the database, and the method includes:

[0137] Step S801, obtaining a regularly set log checkpoint, and taking the log checkpoint as a starting point;

[0138] Step S802, when the system restarts, starting from the starting point, redo operations are performed according to the redo log, rollback operations are performed according to the rollback log, and new rollback operations are determined according to the transaction log to restore the system; wherein the redo log is a write-ahead log of the redo category when modifying data, the rollback log is a write-ahead log that records the rollback category during the rollback process of the current transaction, and the transaction log is a write-ahead log of the transaction category when the current transaction starts, commits, and rolls back.

[0139] It should be noted that the write-ahead log is obtained through the write-ahead log processing method of the above-mentioned database, and the details of the write-ahead log processing are also the same as the write-ahead log processing method of the above-mentioned database, which will not be repeated here.

[0140] In step S801 of some embodiments, the log checkpoint is also called Checkpoint. It should be noted that the data modification operation is to modify the data page in the memory. After each modification, these data pages are not immediately written to the disk. Instead, after a certain period of time, the database engine initiates a checkpoint command to the database. At this time, the command will create a known normal point, write all the pages (dirty pages) that have been modified in the memory, that is, the transaction log information, from the memory to the disk, and record the information about the transaction log. If the database is closed or crashed unexpectedly afterwards, then in the recovery process, the database engine does not need to restore all transaction logs, but starts to apply the changes made in the log from the checkpoint. The data modification operation is to modify the data page in the memory. After each modification, these data pages are not immediately written to the disk. Instead, after a certain period of time, the database engine initiates a checkpoint command to the database. At this time, the command will create a known normal point, write all the pages (dirty pages) that have been modified in the memory, that is, the transaction log information, from the memory to the disk, and record the information about the transaction log. If the database is closed unexpectedly or crashes later, the database engine does not need to restore all transaction logs during the recovery process, but starts the changes made in the write-ahead log from the log checkpoint. Therefore, by setting the log checkpoint, you can determine where to start recovering the system.

[0141] In step S802 of some embodiments, after the system is restarted, the write-ahead log is read according to the log checkpoint as the starting point, and corresponding operations are performed according to the write-ahead log. It should be noted that the read write-ahead log is the write-ahead log recorded when modifying data and transaction operations. Specifically, the write-ahead log that records the redo type when modifying data is defined as a redo log; the transaction start, commit, and rollback operation record transaction type write-ahead log is defined as a transaction log; the write-ahead log that records the rollback type during the transaction rollback process is defined as a rollback log.

[0142] In steps S801 to S802 shown in this embodiment, a log checkpoint is set, and scanning of the write-ahead log is started according to the log checkpoint as a starting point, and corresponding operations are performed according to different types of write-ahead logs to complete system recovery and achieve system recovery.

[0143] Please refer to Fig. 9 In some embodiments, before step S801, the system recovery method may further include: setting a log checkpoint. It should be noted that setting a log checkpoint includes but is not limited to steps S901 to S904:

[0144] Step S901, partitioning the system data buffer to obtain data buffer partitions, and periodically polling to select a data buffer partition, writing all data pages therein to the disk, and taking the minimum value of the global log sequence number of the data page as the global log sequence number of the data buffer partition;

[0145] Step S902, obtaining the minimum value of the global log sequence numbers of all data buffer partitions as the global log sequence number of the system data buffer;

[0146] Step S903, obtaining the minimum value of the global log sequence numbers of all active transactions as the minimum active transaction global log sequence number;

[0147] Step S904: Set the minimum value of the global log sequence number of the system data buffer and the minimum active transaction global log sequence number as a log checkpoint.

[0148] In step S901 of some embodiments, the buffer pool is defined as a buffer pool, the system data buffer is defined as a buffer shard, and the system data buffer is partitioned to obtain data buffer partitions, that is, the buffer pool is divided into S buffer shards, and a data buffer partition is selected from multiple data buffer partitions as the selected data buffer partition. It should be noted that a data buffer partition is selected as the selected data buffer partition by a round-robin scheduling algorithm. After the data buffer partition selection is completed, all data pages (also called dirty pages) in the selected data buffer partition are flushed to the disk, and the dirty page refers to the content that has been modified in the buffer pool. By flushing the modified data pages to the disk, the data in the data buffer partition is transferred to the disk storage, making the data storage more stable.

[0149] In step S902 of some embodiments, the minimum value of the global log sequence number of the data page is used as the global log sequence number of the data buffer partition, and the minimum value of the global log sequence number of the data buffer partition is used as the global log sequence number of the system data buffer. It should be noted that the minimum value of the checkpointed GSN of the buffer shard is used to determine the checkpointed GSN of the entire buffer pool.

[0150] In step S903 of some embodiments, the firstGSN and currentGSN of each active transaction are recorded in the active transaction list, that is, the minimum value of the global log sequence number (that is, the first GSN) among all active transactions is calculated and recorded as minActiveTxGSN to determine the minimum active transaction global log sequence number.

[0151] In step S904 of some embodiments, the minimum value of the global log sequence number of the system data buffer and the minimum active transaction global log sequence number is set as the log checkpoint, that is, log GSN=min{checkpointed GSN,minActiveTxGSN}, log GSN is the log checkpoint, and the pre-written log before the log checkpoint can be cleared, and system recovery is executed from the log checkpoint.

[0152] In steps S901 to S904 shown in this embodiment, the buffer is first partitioned and then the minimum log global sequence number of each partition is determined, and then the log global sequence number of the buffer partition is determined, and then the minimum log global sequence number of the active transaction is determined. Finally, the log checkpoint is determined based on the log global sequence number of the buffer and the minimum log global sequence number of the active transaction to determine from which point to start operating the write-ahead log and complete system recovery.

[0153] Please refer to Fig.10 In some embodiments, before step S802, the system recovery method may also include but is not limited to steps S1001 to S1004:

[0154] Step S1001, starting scanning from a starting point for each log file generated by a plurality of log writing units;

[0155] Step S1002: if a target transaction starts to operate a log file during the process of scanning the log file, the target transaction is added to the active transaction list, and when the target transaction commits and rolls back the write-ahead log, the target transaction is removed from the active transaction list;

[0156] Step S1003, if the write-ahead log encountered in the process of scanning the log file is a redo log and a rollback log, the write-ahead log is grouped according to the modified data page sequence number, the write-ahead logs in the same group are sorted according to the log global sequence number and the number of modification times of the data page, and the data log information of the transaction corresponding to the write-ahead log in the active transaction list is updated;

[0157] Step S1004, obtaining a list of active transactions after scanning all write-ahead logs, and adding all target transactions and data log information corresponding to the target transactions to a rollback list.

[0158] It should be noted that system recovery is divided into three stages: analysis stage, redo stage and rollback stage. The analysis stage is to find the data log information that needs to be rolled back and add the data log information to the rollback list to facilitate subsequent rollback operations.

[0159] Specifically, in the analysis phase: scan each log file starting from the log checkpoint, and scan the checkpoint control file according to logGSN, and scan each log file starting from the log checkpoint. If a transaction start operation log is encountered during the log file scanning process, the transaction is added to the active transaction list, and the transaction is removed from the active transaction list when the transaction commit and rollback completion logs are encountered; if redo logs and rollback logs are encountered during the log file scanning process, they are grouped according to the modified data page sequence number, and the logs that modify the same data page are grouped together, and the pre-written logs of the same data page are sorted according to the log global sequence number and the number of page modifications of the data page; at the same time, the latest data log information of the transaction corresponding to the log in the active transaction list is updated. Finally, the active transaction list after scanning all logs is obtained, and all transaction information and its corresponding latest data log information are added to the rollback list.

[0160] Redo phase: Assign a group of data pages to each log recovery unit, and redo the write-ahead logs of these data pages; redo the sorted redo write-ahead logs in sequence according to the data pages.

[0161] Rollback phase: traverse the rollback list generated in the analysis phase, find the latest write-ahead log of the transaction to be rolled back, if the write-ahead log is a redo log, find the previous rollback log, find all the rollback logs of the transaction from the rollback log in sequence, and roll back; if the log is a rollback log, find all the rollback logs of the transaction from the rollback log in sequence, and roll back.

[0162] Specifically, in the uodo phase, the target update list is traversed, and the last log of the transaction to be undo is found. If it is a CLR, the previous undo log of the corresponding undo log is found and the undo log is traversed in sequence and rollback is performed (the rollback operation will also generate the CLR of the undo log); if it is an undo log, all undo logs are traversed in sequence from this undo log to perform the rollback operation. Therefore, this embodiment performs the rollback operation by writing the pre-log to restore the system and improve the stability of the system.

[0163] See also Fig.11 The embodiment of the present application also provides a write-ahead log processing device for a database, which is applied to a process model of multiple transaction execution units and multiple log writing units, and the multiple transaction execution units execute transactions in parallel, and the log writing units write logs in parallel. The write-ahead log processing device for a database can implement the write-ahead log processing method for the above-mentioned database, and the device includes:

[0164] The data acquisition module 1101 is used to acquire the log write disk associated data of the current transaction; wherein the log write disk associated data includes the current log global sequence number and data page modification information, and the current log global sequence number is the log global sequence number of the current transaction maintained by the transaction execution unit;

[0165] The first transaction processing module 1102 is used to wait for the write-ahead log corresponding to the current transaction to be written to the preset disk by the log writing unit, and for all the write-ahead logs with log global sequence numbers less than the current log global sequence number to be written to the disk by the corresponding log writing unit, and then commit the current transaction if the data page modification information indicates that the current transaction involves cross-data page modification;

[0166] The second transaction processing module 1103 is used to wait for the write-ahead log corresponding to the current transaction to be written to the disk by the corresponding log writing unit if the data page modification information indicates that the current transaction does not involve cross-data page modification, and then commit the current transaction.

[0167] The specific implementation of the write-ahead log processing device for the database is substantially the same as the specific implementation of the write-ahead log processing method for the database described above, and will not be described in detail herein.

[0168] See also Fig.12 The embodiment of the present application further provides a system recovery device, which obtains a write-ahead log by the write-ahead log processing method of the database as described above, and the system recovery device can implement the system recovery method as described above, and the device includes:

[0169] The checkpoint setting module 1201 is used to obtain a log checkpoint that is set regularly and use the log checkpoint as a starting point;

[0170] The recovery module 1202 is used to restore the system from the starting point when the system is restarted, perform redo operations according to the redo log, perform rollback operations according to the rollback log, and determine new rollback operations according to the transaction log; wherein the redo log is a pre-written log of the redo category when modifying data, the rollback log is a pre-written log that records the rollback category during the rollback process of the current transaction, and the transaction log is a pre-written log of the transaction category when the current transaction is started, committed, and rolled back.

[0171] The specific implementation of the system recovery device is substantially the same as the specific implementation of the above-mentioned system recovery method, and will not be described in detail here.

[0172] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the write-ahead log processing method of the above-mentioned database is implemented.

[0173] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0174] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0175] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0176] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0177] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0178] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0179] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0180] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0181] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0182] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0183] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0184] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for processing a write-ahead log of a database, characterized in that: A process model is applied to a plurality of transaction execution units and a plurality of log writing units, wherein the plurality of transaction execution units execute transactions in parallel and the log writing units write logs in parallel, and the method comprises: Obtaining log write disk associated data of the current transaction; wherein the log write disk associated data includes the current log global sequence number and data page modification information, and the current log global sequence number is the log global sequence number of the current transaction maintained by the transaction execution unit; If the data page modification information indicates that the current transaction involves cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be written to the preset disk by the log writing unit, and for all the write-ahead logs whose log global sequence numbers are smaller than the current log global sequence number to be written to the disk by the corresponding log writing unit, and then commit the current transaction; If the data page modification information indicates that the current transaction does not involve cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be written to the disk by the corresponding log writing unit, and then commit the current transaction; Wherein, one of the transaction execution units or one of the log writing units is driven by a thread or a coroutine, each of the transaction execution units is provided with a write-ahead log buffer, each of the log writing units processes the write-ahead log buffers of one or more transaction execution units in parallel, each of the log writing units has its own independent write-ahead log disk storage path, and before obtaining the log writing disk associated data of the current transaction, the method further includes: When the current transaction starts, obtaining log global sequence data of the current transaction; When processing a data page according to a preset operation, the log global sequence data of the current transaction and data page related parameters of the data page are maintained, and a write-ahead log is generated and written into a write-ahead log buffer; wherein changes in the data page related parameters are used to determine the data page modification information; The log global sequence data includes: transaction number, log global sequence number, and sequence number minimum value. When the current transaction starts, obtaining the log global sequence data of the current transaction includes: When the current transaction starts, obtaining the transaction number of the current transaction; Obtain the log global sequence number of the current transaction; The minimum value of the maximum log global sequence number of the current transaction is obtained to obtain the minimum value of the sequence number; wherein the maximum log global sequence number of the current transaction is the maximum log global sequence number of the current transaction flushed by each log writing thread.

2. The method according to claim 1, characterized in that The preset operations include: access, single non-cross-page, single cross-page modification, and when processing the data page according to the preset operation, maintaining the log global sequence data of the current transaction and the data page related parameters of the data page, and generating a write-ahead log to write into the write-ahead log buffer, including: When accessing a data page, setting the current log global sequence number of the current transaction to the maximum value of the current log global sequence number and the log global sequence number of the data page; When a data page is modified in a single non-cross-page manner, the log global sequence number of the data page remains unchanged, and the modification times of the data page are incremented; When a data page is modified across pages in a single time, the current global log sequence number of the current transaction is incremented, the global log sequence number of the data page is set to the current global log sequence number of the current transaction, and the number of modifications of the data page is incremented, and the transaction number of the last modification transaction execution unit of the data page is changed to the transaction number of the current transaction execution unit; Generate a write-ahead log and write it into the write-ahead log buffer.

3. The method according to claim 2, characterized in that After the generated write-ahead log is written into the write-ahead log buffer, the method further includes: The log global sequence number of the write-ahead log record is set to the maximum value of the log global sequence number of the current transaction when modifying the data page and the log global sequence number of the data page.

4. The method according to any one of claims 1 to 3, characterized in that: If the data page modification information indicates that the current transaction involves cross-data page modification, waiting for the write-ahead log corresponding to the current transaction to be written to the preset disk by the log writing unit, and for all the write-ahead logs whose log global sequence numbers are smaller than the current log global sequence number to be written to the disk by the corresponding log writing unit, and then committing the current transaction, includes: If the data page modification information indicates that the current transaction involves cross-data page modification, obtain the maximum value of the log global sequence number of the written logs in each of the log writing units to obtain the log global sequence number of the log writing unit; wherein the written log is a pre-written log that has been written to the disk by the log writing unit; Monitoring the log writing unit where the current transaction to be committed is located to write all pre-written logs corresponding to the current transaction to the disk; Monitor that the log global sequence numbers of all the log writing units are greater than or equal to the current log global sequence number of the current transaction, and commit the current transaction.

5. The method according to claim 4, characterized in that The monitoring that the log global sequence number of all the log writing units is greater than or equal to the current log global sequence number of the current transaction and submitting the current transaction includes: Obtain remote log disk dependency mark information; wherein, at the beginning of the current transaction, obtain the minimum value of the log global sequence number of all log writing units and record it as the visible disk log global sequence number of the current transaction, and set the remote log disk dependency mark information to false; when accessing a data page, if the log global sequence number of the accessed data page is greater than the visible disk log global sequence number of the current transaction, and the transaction number of the last modified transaction execution unit of the data page is different from the transaction number of the current transaction execution unit, then set the remote log disk dependency mark information to true; If the remote log disk dependency mark information is true, monitor the log global sequence numbers of all the log writing units, wait until the log global sequence numbers of all the log writing units are greater than or equal to the current log global sequence number of the current transaction, and monitor the log writing unit where the current transaction to be committed is located to write all the pre-written logs of the current transaction to the disk, and commit the current transaction; If the remote log disk dependency mark information is false, the log disk writing unit where the current transaction to be committed is monitored writes all pre-written logs of the current transaction to the disk, and then commits the current transaction.

6. A system recovery method, characterized in that: The write-ahead log is obtained by the write-ahead log processing method for a database according to any one of claims 1 to 5, the method comprising: Obtain a regularly set log checkpoint, and use the log checkpoint as a starting point; When the system is restarted, starting from the starting point, a redo operation is performed according to the redo log, a rollback operation is performed according to the rollback log, and a new rollback operation is determined according to the transaction log to restore the system; wherein the redo log is a write-ahead log of the redo category when modifying data, the rollback log is a write-ahead log that records the rollback category during the rollback process of the current transaction, and the transaction log is a write-ahead log of the transaction category when the current transaction is started, committed, and rolled back.

7. The method according to claim 6, characterized in that Before obtaining the periodically set log checkpoint and taking the log checkpoint as the starting point, the method further includes: Setting the log checkpoint specifically includes: Partitioning the system data buffer to obtain data buffer partitions, periodically polling and selecting a data buffer partition, writing all data pages therein to a disk, and using the minimum value of the global log sequence number of the data page as the global log sequence number of the data buffer partition; Obtaining the minimum value of the global log sequence numbers of all data buffer partitions as the global log sequence number of the system data buffer; Get the minimum value of the global log sequence number of all active transactions as the minimum active transaction global log sequence number; The minimum value of the global log sequence number of the system data buffer and the minimum active transaction global log sequence number is set as the log checkpoint.

8. The method according to claim 7, characterized in that Before the system is restarted, starting from the starting point, performing a redo operation according to the redo log, performing a rollback operation according to the rollback log, and determining a new rollback operation according to the transaction log to restore the system, the method further includes: Starting scanning from the starting point, a plurality of log writing units generate each log file; If a target transaction starts to operate the log file during scanning the log file, the target transaction is added to the active transaction list, and when the target transaction commits and rolls back the write-ahead log, the target transaction is removed from the active transaction list; If the write-ahead log encountered during scanning of the log file is the redo log and the rollback log, the write-ahead logs are grouped according to the modified data page sequence numbers, the write-ahead logs in the same group are sorted according to the log global sequence number and the number of modification times of the data page, and the data log information of the transaction corresponding to the write-ahead log in the active transaction list is updated; The active transaction list after scanning all write-ahead logs is obtained, and all target transactions and the data log information corresponding to the target transactions are added to a rollback list.

9. A write-ahead log processing device for a database, characterized in that: A process model applied to multiple transaction execution units and multiple log writing units, wherein the multiple transaction execution units execute transactions in parallel, and the log writing units write logs in parallel, wherein the device is used to execute the write-ahead log processing method for a database according to any one of claims 1 to 5, and the device comprises: A data acquisition module, used to acquire log write disk associated data of the current transaction; wherein the log write disk associated data includes the current log global sequence number and data page modification information, and the current log global sequence number is the log global sequence number of the current transaction maintained by the transaction execution unit; A first transaction processing module is configured to, if the data page modification information indicates that the current transaction involves cross-data page modification, wait for the write-ahead log corresponding to the current transaction to be written to a preset disk by the log writing unit, and for all write-ahead logs whose log global sequence numbers are smaller than the current log global sequence number to be written to the disk by the corresponding log writing unit, and then commit the current transaction; The second transaction processing module is used to wait for the write-ahead log corresponding to the current transaction to be written to the disk by the corresponding log writing unit if the data page modification information indicates that the current transaction does not involve cross-data page modification, and then commit the current transaction.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the write-ahead log processing method for a database according to any one of claims 1 to 5 or the system recovery method according to any one of claims 6 to 8 is implemented.

Images