Database log processing method, storage medium and equipment

Abstract

The invention provides a database log processing method, a storage medium and equipment. The database log processing method comprises the steps of obtaining a database log; reading and recording a log serial number of the database log, and recording the log serial number as a reference serial number; obtaining a page data chain table of the database log, and recording the page data chain table as a to-be-stored page data chain table; and storing the to-be-stored page data chain table into a preset global page data chain table according to the ascending order of the log serial numbers according to the reference serial number, so that the page data chain tables in the global page data chain table are sequentially connected according to the ascending order of the log serial numbers. By independently storing the page data chain table in the global page data chain table, the page mirror image data and other data except the page mirror image data can be separately stored subsequently, so that only the other data except the page mirror image data can be synchronized to reduce the data synchronization amount between databases.

Description

Technical Field

[0001] The present invention relates to the technical field of databases, and particularly to a method for processing database logs, a storage medium, and a device. Background Art

[0002] During the use of a database, database logs are recorded. An important function of database logs is to construct a primary and standby database cluster. Briefly, after the primary and standby database cluster is constructed, the primary database provides read and write services externally and continuously generates database logs. The database logs are transmitted to the standby database through streaming replication, so that the standby database is nearly identical to the primary database.

[0003] In order to strictly keep the data of the primary database and the standby database consistent, a synchronous streaming replication mode is usually adopted. In the synchronous streaming replication mode, after the primary database sends a database log to the standby database, it needs to wait for the standby database to store this database log locally or replay it successfully before the primary database can continue to execute. Therefore, the amount of data of the database logs that the primary database needs to transmit to the standby database is an important factor affecting the performance of synchronous streaming replication. Summary of the Invention

[0004] An object of the present invention is to provide a method for processing database logs, a storage medium, and a device that can reduce the amount of data of the database logs to be transmitted.

[0005] Specifically, the present invention provides a method for processing database logs, including:

[0006] Obtain database logs, where the database logs are pre-assembled into a structure composed of an assembly linked list and a page data linked list connected after the assembly linked list. The page data linked list is a linked list composed of page mirror data in the database logs, and the assembly linked list is a linked list composed of other data except the page mirror data in the database logs;

[0007] Read and record the log sequence number of the database logs, denoted as the reference sequence number;

[0008] Obtain the page data linked list of the database logs, denoted as the page data linked list to be stored;

[0009] According to the reference sequence number, store the page data linked list to be stored into a preset global page data linked list in the order of ascending log sequence number, so that the page data linked lists in the global page data linked list are connected in sequence in the order of ascending log sequence number.

[0010] Optionally, the step of storing the page data linked list to be stored into a preset global page data linked list in the order of ascending log sequence number includes:

[0011] Find a suitable position in the global page data linked list for storing the page data linked list to be stored. The suitable position is the position after the page data linked list with the first log sequence number less than the reference sequence number from the back to the front in the global page data linked list, or the position before the page data linked list with the first log sequence number greater than the reference sequence number from the front to the back in the global page data linked list;

[0012] Place the page data linked list to be stored in the suitable position in the global page data linked list.

[0013] Optionally, the steps of storing the page data linked list to be stored in the preset global page data linked list in ascending order of the log sequence number include:

[0014] Before the step of finding a suitable position in the global page data linked list for storing the page data linked list to be stored, check whether the global page data linked list is empty. If so, place the page data linked list to be stored in the global page data linked list; if not, execute the step of finding a suitable position in the global page data linked list for storing the page data linked list to be stored.

[0015] Optionally, the global page data linked list is configured as a doubly linked list. The steps of finding a suitable position in the global page data linked list for storing the page data linked list to be stored include:

[0016] Start searching forward from the end of the global page data linked list;

[0017] Find the page data linked list with the corresponding log sequence number less than the reference sequence number;

[0018] Determine the position after the page data linked list with the log sequence number less than the reference sequence number as the suitable position.

[0019] Optionally, before the steps of storing the page data linked list to be stored in the preset global page data linked list in ascending order of the log sequence number include:

[0020] Obtain the preset lock of the global page data linked list;

[0021] After the steps of storing the page data linked list to be stored in the preset global page data linked list in ascending order of the log sequence number include:

[0022] Release the preset lock of the global page data linked list.

[0023] Optionally, after the step of releasing the preset lock of the global page data linked list include:

[0024] Deposit the assembled linked list part of the database log into the log buffer.

[0025] Optionally, the database log processing method further includes:

[0026] A signal for enabling disk writing of the database log in the cache is detected;

[0027] Obtain the database logs in the cache that meet the preset disk writing rules, denoted as disk-writable logs. The database logs that meet the preset disk writing rules are at least one database log that has been completely written into the cache and whose log sequence numbers are consecutive starting from the log sequence number of the last database log completed in the previous disk writing.

[0028] Find the corresponding page data linked list in the global page data linked list according to the log sequence numbers of the disk-writable logs, denoted as the disk-writable page data linked list;

[0029] Copy the page mirror data in the disk-writable page data linked list to the end of the preset page mirror data storage table in sequence according to the connection order;

[0030] Partially write the assembled linked list of the disk-writable logs to disk.

[0031] Optionally, after the step of copying the page mirror data in the disk-writable page data linked list to the end of the preset page mirror data storage table in sequence according to the connection order, it includes:

[0032] Remove the disk-writable page data linked list from the global page data linked list.

[0033] In another aspect of the present invention, there is also provided a machine-readable storage medium, on which a machine-executable program is stored. When the machine-executable program is executed by a processor, it implements the method for processing the database log according to any one of the above.

[0034] In yet another aspect of the present invention, there is also provided a computer device, including a memory, a processor, and a machine-executable program stored on the memory and running on the processor. When the processor executes the machine-executable program, it implements the method for processing the database log according to any one of the above.

[0035] The method for processing database logs, storage medium, and device of the present invention obtain a database log including an assembled linked list and a page data linked list, read and record the log sequence number of the database log, denoted as the reference sequence number, obtain the page data linked list of the database log, denoted as the page data linked list to be stored, and store the page data linked list to be stored in a preset global page data linked list in ascending order of the log sequence number according to the reference sequence number. That is to say, the page data linked list in the database log is separated from the database log and stored separately in the global page data linked list. That is to say, the page mirror data of the database log and other data except the page mirror data can be written into the cache separately. Then, in the subsequent disk writing process, the page mirror data in the database log and other data except the page mirror data can be disk-written separately. Therefore, when this database synchronizes the database log to other standby databases, it can synchronize only the data except the page mirror data, thereby reducing the data synchronization volume between the primary and standby databases and helping to improve the synchronization efficiency. Moreover, the page mirror data of the database log is still stored on the disk, which can also ensure the security and stability of this database. In addition, by storing the page data linked list to be stored in a preset global page data linked list in ascending order of the log sequence number according to the reference sequence number, the page data linked lists in the global page data linked list are connected in ascending order of the log sequence number. In the subsequent disk writing stage, there is no need to perform additional sorting work on the global page data linked list, reducing the work pressure in the disk writing stage and improving the disk writing efficiency.

[0036] From the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more apparent about the above and other objects, advantages, and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Some specific embodiments of the present invention will be described in detail hereinafter with reference to the accompanying drawings in an exemplary but not restrictive manner. The same reference numerals in the drawings denote the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0038] Figure 1 is a schematic diagram of a database log in the prior art;

[0039] Figure 2 is a schematic flowchart of a method for processing a database log according to an embodiment of the present invention;

[0040] Figure 3 is a first schematic diagram of a database log according to an embodiment of the present invention;

[0041] Figure 4 is a second schematic diagram of a database log according to an embodiment of the present invention;

[0042] Figure 5 It is a schematic diagram of the page data chain representation of the database log according to an embodiment of the present invention;

[0043] Figure 6 It is a schematic diagram of the page mirror data content of the database log according to an embodiment of the present invention;

[0044] Figure 7 It is a third schematic diagram of the database log according to an embodiment of the present invention;

[0045] Figure 8 It is a schematic diagram of the global page data linked list according to an embodiment of the present invention;

[0046] Figure 9 It is a schematic flowchart of the step of storing the page data linked list to be stored into the global page data linked list in the method for processing the database log according to an embodiment of the present invention;

[0047] Figure 10 It is a schematic flowchart of the step of finding a suitable position for storing the page data linked list to be stored in the global page data linked list in the method for processing the database log according to an embodiment of the present invention;

[0048] Figure 11 It is a schematic flowchart of the method for processing the database log according to another embodiment of the present invention;

[0049] Figure 12 It is a schematic flowchart of the method for processing the database log according to yet another embodiment of the present invention;

[0050] Figure 13 It is a schematic diagram of the page mirror data storage table according to an embodiment of the present invention;

[0051] Figure 14 It is a schematic diagram of the machine-readable storage medium according to an embodiment of the present invention;

[0052] Figure 15 It is a schematic diagram of the computer device according to an embodiment of the present invention. Detailed implementation manners

[0053] Those skilled in the art should understand that the embodiments described below are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention. These part of the embodiments are intended to explain the technical principle of the present invention, rather than to limit the protection scope of the present invention. Based on the embodiments provided by the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts should still fall within the protection scope of the present invention.

[0054] It should be noted that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus or device), or used in combination with these instruction execution systems, apparatus or devices.

[0055] The flowchart provided by the present invention is not intended to indicate that the operations of the method will be executed in any specific order, or that all operations of the method are included in every case. In addition, the method may include additional operations. Within the scope of the technical idea provided by the method of this embodiment, additional changes can be made to the above method.

[0056] To facilitate the understanding of this solution, the operation mode of the database in the prior art will be described. First, during the process of modifying the database data, the table stored on the disk is not directly modified. Instead, the data in the table is extracted into the cache in units of data pages (the data to be modified is located in the data pages), modified in the cache, and then written to disk later.

[0057] Furthermore, when modifying the data pages in the cache, database logs will be generated. As Figure 1 shown, the existing database logs generally include two parts: a header and a data area. The data area records various modification operations on the data pages. At the same time, there is also a kind of data in the data area that is page mirror data. The page mirror data is formed after recording the complete page information of a data page in the database log. In the prior art, usually after the log redo point (the log redo point is a log sequence number. Before this log sequence number, the database logs have all been written to disk and the information reflected by the written logs and all the data in the actual database on the disk is the same), if the data of a data page in the cache is modified for the first time, then the entire page information of the data page after the first modification will be recorded in the database log, and then page mirror data will be generated.

[0058] Because during the process of writing a data page to disk, it is possible that a data page crashes during the write process, resulting in a situation where the written data page contains a mixture of old and new data. Therefore, by generating page mirror data, the data page information recorded in the page mirror data can be used to replace the broken page with a mixture of old and new data, and then the subsequent operations recorded in the database log can be replayed to complete the recovery of the data page.

[0059] Therefore, page mirror data is important data for ensuring the security and stability of the database where it is located. However, during the synchronization process of the primary and standby databases, the page mirror data in the database log of the primary database will not affect the data consistency of the primary and standby databases even if it is not synchronized to the standby database. That is to say, if it is possible to separate the page mirror data from the database log of the primary database on the basis of ensuring that the page mirror data of the primary database can be recorded, the amount of data that the primary database needs to synchronize to the standby database can be reduced while ensuring the security and stability of the primary database.

[0060] As Figure 2 shown, in one embodiment, the method for processing a database log generally includes:

[0061] Step S101, obtain a database log. The database log is pre-assembled into a structural page data linked list composed of an assembly linked list and a page data linked list connected after the assembly linked list. The page data linked list is a linked list composed of page mirror data in the database log, and the assembly linked list is a linked list composed of other data in the database log except page mirror data. Specifically, during the assembly process of the database log, the page mirror data of the database log is connected in the form of a page data linked list after the assembly linked list composed of other data of the database log.

[0062] Referring to Figures 3 to 7 shown, specifically, taking a database log with two page mirror data nodes as an example, Figure 3 is an example of the linked list form of an original database log with page mirror data, Figure 4 is Figure 3 an example of the linked list form after removing the page mirror data from the linked list in, where next represents pointing to the next node; len represents the len function for calculating the length of the corresponding data; head data is the header data of the database log; page data is all the page mirror data in the database log; tupledata and main data are other data in the database log except page mirror data. Figure 5 is a simplified example of the linked list of two page mirror data nodes in the database log. wal1 page data1 and wal1 pagedata2 are only simple notations for representing two different page mirror data nodes belonging to the same database log and do not represent specific content. Figure 6An example of the specific content of a page mirror data node. RelFileNode represents the location information of the table to which the data page corresponding to this page mirror data belongs on the disk, and BlockNumber represents the block number of the data page corresponding to this page mirror data in the table to which it belongs. Therefore, RelFileNode and BlockNumber can determine the location of the data page corresponding to the page mirror data on the disk. lsn represents the log sequence number of the database log to which the page mirror data belongs, page data is the complete information of the data page recorded by this page mirror data node, and next points to the next node. Figure 7 An example of the finally formed database log.

[0063] During the assembly process of the database log, first remove the page mirror data in the database log, as Figures 3 to 4 shown. Then form a linked list of page data from the page mirror data in the order in the database log, as Figure 5 shown. Then, connect the linked list of page data behind the linked list composed of other data in the database log, thereby assembling the database log into a structure composed of an assembled linked list and a linked list of page data connected after the assembled linked list, as Figure 7 shown. That is to say, Figure 6 can be regarded as Figure 5 a schematic diagram of the specific content of a page mirror data node in. Figure 7 The assembled linked list in represents Figure 4 the linked list of the database log in, that is, the linked list composed of other data in the database log except for the page mirror data.

[0064] Step S102, read and record the log sequence number of the database log, denoted as the reference sequence number. The log sequence number is incrementally configured during the formation of the database log and can identify the order of the database log. After obtaining the database log, read and record the log sequence number of this database log, denoted as the reference sequence number.

[0065] Step S103, obtain the linked list of page data of the database log, denoted as the linked list of page data to be stored. Specifically, remove the linked list of page data in the database log from the database log. Refer to Figure 7 shown, that is, remove the part of the linked list after connecting the assembled linked list, and the removed part is the linked list of page data to be stored. In simple terms, it is Figure 5 the linked list composed of the page mirror data of this database log shown.

[0066] Step S104, store the to-be-stored page data linked list into a preset global page data linked list in ascending order of the log sequence number according to the reference sequence number, so that the page data linked lists in the global page data linked list are connected in ascending order of the log sequence number. Specifically, the global page data linked list is located in the shared memory, that is, the cache. After removing the page data linked list of the database log, the page data linked list is stored in the global page data linked list, that is, the to-be-stored page data linked list is stored in the global page data linked list. And when storing, the to-be-stored page data linked list is stored in ascending order of the log sequence number. In other words, the global page data linked list is a linked list composed of page data linked lists, and the page data linked lists in the global page data linked list are connected in ascending order of the log sequence number.

[0067] Refer to Figure 8 As shown, it is a simple schematic of the global page data linked list storing two page data linked lists. page_data_list_local represents the page data linked list of a database log, and page_data_list_global represents the global page data linked list. Among them, wal1 and wal2 are only used to indicate that the two page data linked lists belong to different database logs.

[0068] As more and more page data linked lists are stored in the global page data linked list, the global page data linked list will become longer and longer. And if there are already other page data linked lists in the global page data linked list when storing the to-be-stored page data linked list, the to-be-stored page data linked list cannot be simply connected after the existing global page data linked list. It is necessary to store the to-be-stored page data linked list in the correct position in the global page data linked list according to the log sequence number of the database log to which the to-be-stored page data linked list belongs, that is, the reference sequence number, in ascending order of the log sequence number.

[0069] Refer to Figure 9 As shown, specifically, the steps of storing the to-be-stored page data linked list in ascending order of the log sequence number according to the reference sequence number into a preset global page data linked list include:

[0070] Step S201, check whether the global page data linked list is empty. If it is, execute Step S202. If not, execute Step S203. Specifically, that is to check whether the global page data linked list already stores a page data linked list.

[0071] Step S202, put the to-be-stored page data linked list into the global page data linked list. Specifically, if the global page data linked list is empty, that is, there is not a single page data linked list in the global page data linked list, then there is no need to worry about the order problem after putting the to-be-stored page data linked list into the global page data linked list. So the to-be-stored page data linked list can be directly put into the global page data linked list.

[0072] Step S203, find a suitable position in the global page data linked list for storing the to-be-stored page data linked list. The suitable position is the position after the page data linked list with the first log sequence number less than the reference sequence number from the back to the front in the global page data linked list, or the position before the page data linked list with the first log sequence number greater than the reference sequence number from the front to the back in the global page data linked list.

[0073] Specifically, if the global page data linked list is not empty, that is, there are already page data linked lists stored in the global page data linked list. At this time, before putting the to-be-stored page data linked list into the global page data linked list, it is necessary to consider the order relationship between the log sequence number corresponding to the to-be-stored page data linked list and the log sequence numbers of the page data linked lists already in the global page data linked list, so as to put the to-be-stored page data linked list in the correct position. Therefore, first find a suitable position in the global page data linked list for storing the to-be-stored page data linked list.

[0074] Preferably, the global page data linked list is configured as a doubly linked list. On this basis, refer to Figure 10 As shown, this step includes:

[0075] Step S301, start searching forward from the end of the global page data linked list. Specifically, that is, search forward from the last page data linked list in the global page data linked list.

[0076] Step S302, find the page data linked list with the corresponding log sequence number less than the reference sequence number. Specifically, during the process of searching forward from the end of the global page data linked list, once a page data linked list with a log sequence number less than the reference sequence number is found, that is, the log sequence number of the database log to which the found page data linked list belongs is less than the log sequence number of the database log to which the to-be-stored page data linked list belongs.

[0077] Step S303, determine the position after the page data linked list with the log sequence number less than the reference sequence number as the suitable position. Specifically, after finding the page data linked list with the corresponding log sequence number less than the reference sequence number, it indicates that the to-be-stored page data linked list should be connected immediately after this page data linked list. That is to say, the position after this page data linked list is the suitable position.

[0078] It should be noted that during the process of searching forward from the end of the global page data linked list, if no page data linked list with a log sequence number less than the reference sequence number is found after searching all the page mirror data in the global page data linked list, then store the to-be-stored page data linked list at the front of the global page data linked list.

[0079] It should be noted that when the global page data linked list is a doubly linked list, it can be searched from back to front or from front to back. Specifically, that is, searching from the front of the global page data linked list backwards. Once a log sequence number corresponding to a page data linked list is found to be greater than the reference sequence number, the position before this page data linked list is the appropriate position. Additionally, in some other embodiments, the global page data linked list can also be a singly linked list. In this case, it can only be searched from front to back.

[0080] Step S204, store the page data linked list to be stored in the appropriate position in the global page data linked list. Specifically, after finding the appropriate position belonging to the page data linked list to be stored in the global page data linked list, storing the page data linked list to be stored in this appropriate position can ensure that all the page data linked lists stored in the global page data linked list are connected in sequence according to the ascending order of the log sequence numbers of their respective database logs.

[0081] In the solution of this embodiment, by obtaining the database log including the assembled linked list and the page data linked list, reading and recording the log sequence number of the database log, denoted as the reference sequence number, obtaining the page data linked list of the database log, denoted as the page data linked list to be stored, and storing the page data linked list to be stored in the preset global page data linked list in the ascending order of the log sequence numbers according to the reference sequence number. That is, separating the page data linked list in the database log from the database log and storing it separately in the global page data linked list. That is, the page mirror data of the database log and other data except the page mirror data can be written into the cache separately. Then, during the subsequent disk writing process, the page mirror data in the database log and other data except the page mirror data can be disk-written separately. Therefore, when this database synchronizes the database log to other standby databases, it can synchronize only the data except the page mirror data, thereby reducing the data synchronization volume between the primary and standby databases and helping to improve the synchronization efficiency. Moreover, the page mirror data of the database log is still stored on the disk, which can also ensure the security and stability of this database.

[0082] Additionally, by storing the page data linked list to be stored in the preset global page data linked list in the ascending order of the log sequence numbers according to the reference sequence number, the page data linked lists in the global page data linked list are connected in sequence according to the ascending order of the log sequence numbers. During the subsequent disk writing stage, there is no need to perform additional sorting work on the global page data linked list, reducing the working pressure in the disk writing stage and improving the disk writing efficiency.

[0083] Furthermore, since the log sequence numbers of database logs generated later are larger, for the list of page data to be stored in the global page data linked list, it is very likely to be located at the rear position of the global page data linked list. By configuring the global page data linked list as a doubly linked list and searching forward from the end of the global page data linked list during the process of finding a suitable position in the global page data linked list to store the list of page data to be stored, it helps to improve the search speed.

[0084] As Figure 11 shown, in one embodiment, the method for processing database logs generally includes:

[0085] Step S401, obtain the database log. The database log is pre-assembled into a structural page data linked list consisting of an assembly linked list and a page data linked list connected after the assembly linked list.

[0086] Step S402, read and record the log sequence number of the database log, denoted as the reference sequence number.

[0087] Step S403, obtain the page data linked list of the database log, denoted as the list of page data to be stored.

[0088] Step S404, obtain the preset lock of the global page data linked list. Specifically, the global page data linked list is configured with a mutex lock. During the process of storing the list of page data to be stored in the global page data linked list, first obtain the mutex lock of the global page data linked list, so that only the process that obtains the mutex lock can execute the task of storing the list of page data to be stored in the global page data linked list, avoiding chaos in the sorting of the page data linked list in the global page data linked list.

[0089] Step S405, store the list of page data to be stored in the preset global page data linked list in ascending order of the log sequence number according to the reference sequence number, so that the page data linked lists in the global page data linked list are connected in ascending order of the log sequence number.

[0090] Step S406, release the preset lock of the global page data linked list. Specifically, when the storage of the list of page data to be stored in the global page data linked list is completed, release the mutex lock of the global page data linked list, so that subsequent page data linked lists can continue to be stored in the global page data linked list.

[0091] In the solution of this embodiment, by obtaining the preset lock of the global page data linked list before the step of storing the list of page data to be stored in the preset global page data linked list in ascending order of the log sequence number, it is ensured that before one list of page data to be stored completes its storage in the global page data linked list, other page data linked lists cannot be stored in the global page data linked list, avoiding chaos in the sorting of the page data linked list in the global page data linked list.

[0092] Further, after the step of releasing the preset lock of the global page data linked list, the following steps are included:

[0093] Step S407, storing the assembled linked list part of the database log into the log buffer area. Specifically, allocate a storage location in the log buffer area according to the length of the assembled linked list part, that is, allocate a storage location in the log buffer area according to the length of the database log after removing the page data linked list. Then store the assembled linked list part of the database log into the location allocated for it in the log buffer area. So far, an entire database log has been written into the cache, that is, both the assembled linked list part and the page data linked list part have been written into the cache.

[0094] It should be noted that there is no strict sequence relationship between this step and the step of storing the page data linked list into the global page data linked list.

[0095] As Figure 12 shown, in one embodiment, the method for processing a database log generally further includes:

[0096] Step S501, detecting a start signal for flushing the database log in the cache to disk. Specifically, the start signal can be a signal that is triggered every fixed time, or it can also be a signal that is triggered after the amount of database log data in the cache reaches a certain threshold, etc. Simply put, it is to detect that it is necessary to perform the work of flushing the database log in the cache to disk.

[0097] Step S502, obtaining the database logs in the cache that meet the preset flushing rules, which are recorded as flushable logs. Specifically, the database logs that meet the preset flushing rules are at least one database log that has been completely written into the cache and whose log sequence numbers are consecutive starting from the log sequence number of the last database log completed in the previous flushing. A database log that has been completely written into the cache is a database log whose assembled linked list part has been written into the log buffer area and whose page data linked list has been stored in the global page data linked list.

[0098] Specifically, the preset flushing rules can be configured to take all database logs whose log sequence numbers are consecutive starting from the log sequence number of the last database log completed in the previous flushing as the database logs that can be flushed this time, or it can also be configured to take a set number (such as one, two, ten, one hundred, etc.) of database logs whose log sequence numbers are consecutive starting from the log sequence number of the last database log completed in the previous flushing as the database logs that can be flushed this time.

[0099] Exemplarily, if the database logs with log sequence numbers 3, 4, 5, 8, 9 (the numbers are only for illustration and do not represent the actual complete log sequence numbers) are fully stored in the current cache, and the log sequence number of the last database log completed for disk write last time is 2.

[0100] With a preset disk write rule configured to regard all database logs with consecutive log sequence numbers starting from the log sequence number of the last database log completed for disk write last time as the database logs that can be written to disk this time: starting from 2, 3, 4, 5 are the database logs with consecutive log sequence numbers. There are 6 and 7 missing between 5 and 8, so they are no longer consecutive. Then the database logs with log sequence numbers 3, 4, 5 conform to the preset disk write rule.

[0101] With a preset disk write rule configured to regard two database logs with consecutive log sequence numbers starting from the log sequence number of the last database log completed for disk write last time as the database logs that can be written to disk this time: starting from 2, 3, 4 are the database logs with consecutive log sequence numbers.

[0102] Step S503, find the corresponding page data linked list from the global page data linked list according to the log sequence number of the logs that can be written to disk, and denote it as the page data linked list that can be written to disk. Specifically, after determining the logs that can be written to disk, find the corresponding page data linked list from the global page data linked list according to the log sequence number.

[0103] Step S504, copy the page mirror data in the page data linked list that can be written to disk to the end of a preset page mirror data storage table in the connection order. Specifically, a page mirror data storage table is pre-created on the disk. After obtaining the page data linked list that can be written to disk, copy the page mirror data in the page data linked list that can be written to disk to the page mirror data storage table in the connection order.

[0104] Refer to Figure 13 As shown, it is an example of a page mirror data storage table. Since the page data linked list was stored in the global page data linked list in the order of the log sequence number before, directly copy the page mirror data in the page data linked list that can be written to disk to the end of the storage table in the connection order. The page mirror data must also be arranged in the order of the log sequence number in the page mirror data storage table.

[0105] Step S505, partially write the assembled linked list of the logs that can be written to disk to disk. Specifically, that is, find the assembled linked list part of the database log that needs to be written to disk from the log cache area according to the log sequence number, and then write the found assembled linked list part that needs to be written to disk to disk. Thus, the page data linked list and the assembled linked list of the database log that need to be written to disk in this disk write operation are both written to disk, that is, the complete database log is written to disk.

[0106] It should be noted that there is no strict sequence relationship between this step and the step of putting page mirror data into the page mirror data storage table.

[0107] Further, after the step of sequentially copying the page mirror data in the disk-writable page data linked list to the last part of the preset page mirror data storage table, the following steps are included:

[0108] Step S506, removing the disk-writable page data linked list from the global page data linked list. Specifically, it is to delete the page data linked list that has completed disk writing in the global page data linked list, so as to free up storage space.

[0109] In the solution of this embodiment, by storing the page mirror data in the page mirror data storage table, since the database can manage and search the table more conveniently, the management convenience of the page mirror data on the disk can be improved. And because the page data linked list was arranged in the order of log sequence numbers during the process of storing it in the global page data linked list before, the page mirror data can be directly put into the last part of the page mirror data storage table without sorting during the disk writing stage, and the efficiency is very high.

[0110] This embodiment also provides a machine-readable storage medium and a computer device. Figure 14 It is a schematic diagram of a machine-readable storage medium 10 according to an embodiment of the present invention. Figure 15 It is a schematic diagram of a computer device 20 according to an embodiment of the present invention.

[0111] The machine-readable storage medium 10 stores a machine-executable program 11 thereon. When the machine-executable program 11 is executed by a processor, it implements the processing method of the database log in any of the above embodiments.

[0112] The computer device 20 may include a memory 210, a processor 220, and a machine-executable program 11 stored on the memory 210 and running on the processor 220. When the processor 220 executes the machine-executable program 11, it implements the processing method of the database log in any of the above embodiments.

[0113] For the description of this embodiment, the machine-readable storage medium 10 can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the machine-readable storage medium 10 can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0114] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiment, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0115] The computer device 20 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smart phone. In some examples, the computer device 20 can be a cloud computing node. The computer device 20 can be described in the general context of computer system-executable instructions, such as program modules, executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer device 20 can be implemented in a distributed cloud computing environment where tasks are executed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0116] The computer device 20 can include a processor 220 adapted to execute stored instructions and a memory 210 that provides temporary storage space for the operation of the instructions during operation. The processor 220 can be a single-core processor, a multi-core processor, a computing cluster, or any other number of other configurations. The memory 210 can include a random access memory (RAM), a read-only memory, a flash memory, or any other suitable storage system.

[0117] The processor 220 may be connected through a system interconnect (such as PCI, PCI-Express, etc.) to an I / O interface (input / output interface) suitable for connecting the computer device 20 to one or more I / O devices (input / output devices). The I / O devices may include, for example, a keyboard and a pointing device, where the pointing device may include a touchpad or a touch screen, etc. The I / O devices may be built-in components of the computer device 20 or may be devices externally connected to the computing device.

[0118] The processor 220 may also be linked through a system interconnect to a display interface suitable for connecting the computer device 20 to a display device. The display device may include a display screen as a built-in component of the computer device 20. The display device may also include a computer monitor, a television, a projector, etc. externally connected to the computer device 20. In addition, a network interface controller (NIC) may be suitable for connecting the computer device 20 to a network through a system interconnect. In some embodiments, the NIC may use any suitable interface or protocol (such as Internet Small Computer System Interface, etc.) to transmit data. The network may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, etc. A remote device may be connected to the computing device through the network.

[0119] At this point, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications consistent with the principles of the present invention can still be directly determined or derived from the disclosed content of the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and determined to cover all these other variations or modifications.

Claims

1. A method for processing database logs, comprising: Obtaining a database log, where the database log is pre-assembled into a structure composed of an assembly linked list and a page data linked list connected after the assembly linked list. The page data linked list is a linked list composed of page mirror data in the database log, and the assembly linked list is a linked list composed of other data except page mirror data in the database log; Reading and recording the log sequence number of the database log, denoted as the reference sequence number; Obtaining the page data linked list of the database log, denoted as the to-be-stored page data linked list; Storing the to-be-stored page data linked list into a preset global page data linked list in ascending order of the log sequence number according to the reference sequence number, so that the page data linked lists in the global page data linked list are connected in ascending order of the log sequence number.

2. The method for processing database logs according to claim 1, wherein, The step of storing the to-be-stored page data linked list into a preset global page data linked list in ascending order of the log sequence number includes: Finding a suitable position in the global page data linked list for storing the to-be-stored page data linked list. The suitable position is the position after the page data linked list with the first log sequence number less than the reference sequence number from the back to the front in the global page data linked list, or the position before the page data linked list with the first log sequence number greater than the reference sequence number from the front to the back in the global page data linked list; Putting the to-be-stored page data linked list into the suitable position in the global page data linked list.

3. The method for processing a database log according to claim 2, wherein, The step of storing the to-be-stored page data linked list into a preset global page data linked list in ascending order of the log sequence number includes: Before the step of finding a suitable position in the global page data linked list for storing the to-be-stored page data linked list, checking whether the global page data linked list is empty. If it is, putting the to-be-stored page data linked list into the global page data linked list; if not, executing the step of finding a suitable position in the global page data linked list for storing the to-be-stored page data linked list.

4. The method for processing database logs according to claim 2, wherein, The global page data linked list is configured as a doubly linked list. The step of finding a suitable position in the global page data linked list for storing the to-be-stored page data linked list includes: Starting from the end of the global page data linked list and searching forward; Finding the page data linked list with the corresponding log sequence number less than the reference sequence number; Determining the position after the page data linked list with the log sequence number less than the reference sequence number as the suitable position.

5. The method for processing a database log according to claim 1, wherein, Before the step of storing the to-be-stored page data linked list into a preset global page data linked list in ascending order of the log sequence number includes: Obtaining the preset lock of the global page data linked list; After the step of storing the to-be-stored page data linked list into a preset global page data linked list in ascending order of the log sequence number includes: Releasing the preset lock of the global page data linked list.

6. The method for processing a database log according to claim 5, wherein, After the step of releasing the preset lock of the global page data linked list includes: Storing the assembly linked list part of the database log into the log buffer.

7. The method for processing database logs according to claim 1, wherein, The method for processing the database log further includes: An enabling signal for flushing the database log in the cache is detected; Obtain the database logs in the cache that meet the preset flushing rules, denoted as flushing-enabled logs. The database logs that meet the preset flushing rules are at least one database log that has been completely written into the cache and whose log sequence numbers are consecutive starting from the log sequence number of the last database log completed in the previous flushing; Find the corresponding page data linked list in the global page data linked list according to the log sequence numbers of the flushing-enabled logs, denoted as the flushing-enabled page data linked list; Copy the page mirror data in the flushing-enabled page data linked list to the end of a preset page mirror data storage table in sequence according to the connection order; Partially flush the assembled linked list of the flushing-enabled logs.

8. The method for processing a database log according to claim 7, wherein, After the step of copying the page mirror data in the flushing-enabled page data linked list to the end of a preset page mirror data storage table in sequence according to the connection order, it includes: Remove the flushing-enabled page data linked list from the global page data linked list.

9. A machine-readable storage medium, on which a machine-executable program is stored. When the machine-executable program is executed by a processor, it implements the method for processing database logs according to any one of claims 1 to 8.

10. A computer device, including a memory, a processor, and a machine-executable program stored on the memory and running on the processor. When the processor executes the machine-executable program, it implements the method for processing database logs according to any one of claims 1 to 8.

Images