Transaction log processing method of database, storage medium and equipment

Abstract

The invention provides a transaction log processing method of a database, a storage medium and equipment. The page mirror data of the transaction log is stored in a storage file configured in a disk. The transaction log processing method of the database comprises the steps of obtaining table position information of a table to which a data page corresponding to page mirror image data to be searched belongs in a disk, a block number of the data page corresponding to the page mirror image data to be searched in the table to which the data page belongs, and a log serial number of a transaction log to which the page mirror image data to be searched belongs, respectively recording as a target table position, a target block number and a target serial number; traversing storage files in the disk; obtaining a storage file of which the table position information in the file name is the same as the target table position in the disk, and recording the storage file as a preliminary target file; and obtaining page mirror image data with the same block number as the target block number and the same log serial number as the target serial number in the preliminary target file. The convenience of using the transaction log is improved from multiple aspects, and the operation efficiency of the database is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of databases, and particularly to a method for processing transaction logs of a database, a storage medium, and a device. Background Art

[0002] During the use of a database, transaction logs are recorded. The transaction logs record historical operation information of the database data and are important data of the database. Transaction logs are essential data for various tasks of the database, such as fault recovery and primary and standby database construction. However, in the prior art, the use of transaction logs in a database is relatively inconvenient, affecting the processing efficiency of the database. Summary of the Invention

[0003] An object of the present invention is to provide a method for processing transaction logs of a database, a storage medium, and a device that can improve the convenience of using transaction logs.

[0004] Specifically, the present invention provides a method for processing transaction logs of a database. The page mirror data of the transaction logs is stored in a storage file configured on a disk. The storage file stores page mirror data corresponding to at least one data page belonging to the same table. The file name of the storage file records the table location information of the table to which the data page corresponding to the page mirror data stored in the storage file belongs on the disk. And the method for processing transaction logs of the database includes:

[0005] Obtain the table location information of the table to which the data page corresponding to the page mirror data to be searched belongs on the disk, the block number of the data page corresponding to the page mirror data to be searched in the belonging table, and the log sequence number of the transaction log to which the page mirror data to be searched belongs, and record them as the target table location, the target block number, and the target sequence number respectively;

[0006] Traverse the storage files on the disk;

[0007] Obtain the storage file on the disk whose table location information in the file name is the same as the target table location, and record it as the preliminary target file;

[0008] Obtain the page mirror data in the preliminary target file whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number.

[0009] Optionally, before the step of obtaining the page mirror data in the preliminary target file whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number, it includes:

[0010] Detect whether the preliminary target file stores only one page mirror data;

[0011] If so, detect whether one of the page mirror data in the preliminary target file has the same block number as the target block number and the same log sequence number as the target sequence number. If so, perform the step of obtaining the page mirror data in the preliminary target file that has the same block number as the target block number and the same log sequence number as the target sequence number.

[0012] Optionally, after the step of detecting whether one of the page mirror data in the preliminary target file has the same block number as the target block number and the same log sequence number as the target sequence number, it includes:

[0013] If not, determine whether the traversal of the storage files in the disk is completed. If so, determine that the page mirror data to be searched does not exist in the disk. If not, continue to perform the step of traversing the storage files in the disk.

[0014] Optionally, after the step of detecting whether the preliminary target file stores only one page mirror data, it includes:

[0015] If not, sequentially read the page mirror data in the preliminary target file, and detect whether the read page mirror data has the same block number as the target block number and the same log sequence number as the target sequence number. If so, perform the step of obtaining the page mirror data in the preliminary target file that has the same block number as the target block number and the same log sequence number as the target sequence number;

[0016] If not, determine whether all the page mirror data in the preliminary target file have been read. If so, perform the step of determining whether the traversal of the storage files in the disk is completed. If not, continue to perform the step of sequentially reading the page mirror data in the storage file.

[0017] Optionally, the file name of the storage file storing multiple page mirror data also records the maximum block number and the minimum block number of all the page mirror data stored in the storage file, as well as the maximum log sequence number and the minimum log sequence number;

[0018] After the step of detecting whether the preliminary target file stores only one page mirror data, it includes:

[0019] If not, determine whether the target block number is greater than or equal to the minimum block number and less than or equal to the maximum block number, and the target sequence number is greater than or equal to the minimum log sequence number and less than or equal to the maximum log sequence number. If so, further search the preliminary target file. If not, perform the step of determining whether the traversal of the storage files in the disk is completed.

[0020] Optionally, all the page mirror data in the storage file storing multiple page mirror data are arranged in ascending order of block number;

[0021] The step of further searching for the preliminary target file includes:

[0022] Obtain the differences between the target block number and the minimum block number, and between the maximum block number and the target block number;

[0023] If the difference between the target block number and the minimum block number is greater than the difference between the maximum block number and the target block number, start searching from the last page mirror data of the preliminary target file;

[0024] If the difference between the target block number and the minimum block number is less than or equal to the difference between the maximum block number and the target block number, start searching from the first page mirror data of the preliminary target file.

[0025] Optionally, a storage file storing a plurality of page mirror data is configured with a file header, and the content of the file header includes the number of page mirror data in the storage file, a block number array, a log sequence number array, and a starting address array. Among them, the block number array stores the block numbers of all page mirror data in the storage file, the log sequence number array stores the log sequence numbers of all page mirror data in the storage file, and the starting address array stores the starting addresses of all page mirror data in the storage file;

[0026] The step of further searching for the preliminary target file includes:

[0027] Read the file header;

[0028] Traverse the block number array and the log sequence number array;

[0029] Detect whether there are block numbers and log sequence numbers at the same position that are both the same as the target block number and the target sequence number. If so, execute the step of obtaining the page mirror data in the preliminary target file with the block number the same as the target block number and the log sequence number the same as the target sequence number. If not, execute the step of determining whether the traversal of the storage file on the disk is completed.

[0030] Optionally, the step of reading the file header includes:

[0031] Calculate the length of the file header according to the header length calculation formula;

[0032] Among them, the header length calculation formula is:

[0033] header_len = sizeof(N)+(sizeof(BlockNumber[0])+sizeof(lsn[0])+sizeof(ptr[0]))*N;

[0034] header_len is the length of the file header, sizeof(N) is the length of the field in the file header that records the number of page mirror data, BlockNumber[0] is the length of the field of the first block number in the block number array, lsn[0] is the length of the field of the first log sequence number in the log sequence number array, ptr[0] is the length of the field of the first first address in the first address array, and N is the number of page mirror data in the preliminary target file;

[0035] The file header is read according to the length of the file header.

[0036] In another aspect, the present invention further provides a machine-readable storage medium having a machine executable program stored thereon, wherein the machine executable program, when executed by a processor, implements the transaction log processing method for a database according to any one of the above items.

[0037] In yet another aspect, the present invention further provides a computer device comprising a memory, a processor, and a machine executable program stored in the memory and running on the processor, and the processor implements the transaction log processing method of the database according to any one of the above items when executing the machine executable program.

[0038] The transaction log processing method, storage medium and device of the database of the present invention store the page mirror data of the transaction log in a storage file in a disk, and record the table location information of the table in the disk to which the data page corresponding to the page mirror data stored in the storage file belongs in the file name of the storage file. On the one hand, the page mirror data of the transaction log and other data except the page mirror data are stored separately in the disk, so that when the database synchronizes the transaction log with other standby databases, only other data except the page mirror data can be synchronized, thereby reducing the data synchronization amount between the primary and standby databases, which helps to improve the synchronization efficiency. On the other hand, by obtaining the table position information of the table to which the data page corresponding to the page mirror data to be found belongs in the disk, the block number of the data page corresponding to the page mirror data to be found in the table to which it belongs, and the log sequence number of the transaction log to which the page mirror data to be found belongs, respectively recorded as the target table position, the target block number and the target sequence number, traversing the storage files in the disk; obtaining the storage file whose table position information in the file name in the disk is the same as the target table position, recorded as the preliminary target file; obtaining the page mirror data in the preliminary target file whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number, that is, when searching for page mirror data, only the storage file needs to be searched, and the page mirror data can be found more quickly than searching for the complete transaction log with page mirror data. Therefore, the convenience of using transaction logs is improved from multiple aspects, which helps to improve the operation efficiency of the database.

[0039] Upon a detailed description of specific embodiments of the present invention below in conjunction with the accompanying drawings, those skilled in the art will become more clearly aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Some specific embodiments of the present invention will be described in detail hereinafter with reference to the accompanying drawings in an exemplary but not restrictive manner. Identical reference numerals in the drawings denote identical or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0041] Figure 1 is a schematic diagram of a transaction log in the prior art;

[0042] Figure 2 is a first schematic diagram of a transaction log according to an embodiment of the present invention;

[0043] Figure 3 is a second schematic diagram of a transaction log according to an embodiment of the present invention;

[0044] Figure 4 is a schematic diagram of a page data chain representation of a transaction log according to an embodiment of the present invention;

[0045] Figure 5 is a schematic diagram of the page mirror data content of a transaction log according to an embodiment of the present invention;

[0046] Figure 6 is a third schematic diagram of a transaction log according to an embodiment of the present invention;

[0047] Figure 7 is a schematic flowchart of a transaction log processing method according to an embodiment of the present invention;

[0048] Figure 8 is a schematic flowchart of a transaction log processing method according to another embodiment of the present invention;

[0049] Figure 9 is a schematic diagram of a storage file having multiple page mirror data according to an embodiment of the present invention;

[0050] Figure 10 is a schematic flowchart of a transaction log processing method according to yet another embodiment of the present invention;

[0051] Figure 11 is a schematic diagram of a storage file having a file header according to an embodiment of the present invention;

[0052] Figure 12 is a schematic flowchart of the step of further searching for a preliminary target file in a transaction log processing method according to yet another embodiment of the present invention;

[0053] Figure 13 Schematic diagram of a machine-readable storage medium according to an embodiment of the present invention;

[0054] Figure 14 Schematic diagram of a computer device according to an embodiment of the present invention. Detailed implementation manners

[0055] Those skilled in the art should understand that the embodiments described below are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention. This part of the embodiments is intended to explain the technical principle of the present invention, rather than to limit the protection scope of the present invention. Based on the embodiments provided by the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts should still fall within the protection scope of the present invention.

[0056] It should be noted that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or used in combination with these instruction execution systems, apparatus, or devices.

[0057] The flowchart provided by the present invention is not intended to indicate that the operations of the method will be executed in any specific order, or that all operations of the method are included in every case. In addition, the method may include additional operations. Within the scope of the technical idea provided by the method of this embodiment, additional changes can be made to the above method.

[0058] To facilitate understanding of this solution, the operation mode of the database in the prior art will be described. First, during the process of modifying the database data, the table stored on the disk is not directly modified. Instead, the data in the table is extracted into the cache in units of data pages (the data to be modified is located in the data pages), modified in the cache, and then written to disk later.

[0059] Furthermore, when modifying the data pages in the cache, transaction logs will be generated. As Figure 1As shown, the structural form of the existing transaction log generally includes two parts: a header and a data area. The data area records various modification operations on data pages. At the same time, there is a kind of data in the data area that is page mirror data, and the page mirror data is the data formed after recording the complete page information of a data page into the transaction log. In the prior art, usually after the log redo point (the log redo point is a log sequence number, and before this log sequence number, the transaction log has been flushed to disk and the information reflected by the flushed log and all the data in the actual database on the disk is the same), if the data of a data page in the cache is modified for the first time, then the entire page information of the data page after the first modification will be recorded in the transaction log, and then page mirror data is generated. The prior art writes the transaction log into the cache and flushes it to disk in the Figure 1 structural form.

[0060] As Figures 2 to 6 shown, in an embodiment of the present application, the structural form of the transaction log is modified so that the page mirror data of the transaction log is stored in a storage file on the disk. Specifically, during the generation process of the transaction log, for a transaction log with page mirror data in its content, when all the content of a transaction log is determined, the transaction log is assembled into a structure composed of an assembly linked list and a page data linked list. The page data linked list is a linked list composed of the page mirror data in the transaction log, and the assembly linked list is a linked list composed of other data in the transaction log except the page mirror data.

[0061] Referring to Figures 2 to 6 shown, specifically, taking a transaction log with two page mirror data nodes in the complete content as an example, Figure 2 is an example of the linked list form of the original transaction log with page mirror data, Figure 3 is Figure 2 an example of the linked list form after removing the page mirror data from the linked list in Figure 4 is a simplified example of the linked list of two page mirror data nodes in the transaction log. wal1 page data1 and wal1 pagedata2 are only simple notations used to represent two different page mirror data nodes belonging to the same transaction log, and do not represent specific content. Figure 5An example of the specific content of a page mirror data node. RelFileNode represents the location information of the table to which the data page corresponding to this page mirror data belongs on the disk, and BlockNumber represents the block number of the data page corresponding to this page mirror data in the table to which it belongs. Therefore, RelFileNode and BlockNumber can determine the location of the data page corresponding to the page mirror data on the disk. lsn represents the log sequence number of the transaction log to which the page mirror data belongs, page data is the complete information of the data page recorded by this page mirror data node, and next points to the next node. Figure 6 An example of the finally formed transaction log.

[0062] During the assembly process of the transaction log, first remove all page mirror data in the transaction log, as Figures 2 to 3 shown. Then form a linked list of page data from the page mirror data in the order in the transaction log, as Figure 4 shown. Then, connect the linked list of page data to the rear of the linked list composed of other data in the transaction log, thereby assembling the transaction log into a structure composed of an assembly linked list and a linked list of page data connected after the assembly linked list, as Figure 6 shown. That is to say, Figure 5 can be regarded as Figure 4 a schematic diagram of the specific content of a page mirror data node in Figure 6 The assembly linked list in Figure 3 represents the linked list of the transaction log in

[0063] That is, the linked list of other data in the transaction log except for the page mirror data.

[0064] Therefore, the structural form of the transaction log becomes the form of an assembly linked list and a linked list of page data connected after it.

[0065] During the process of writing the transaction log into the cache, write the assembly linked list and the linked list of page data separately into the cache. Then, when the transaction log is written to disk, the page mirror data of the transaction log and the data other than the page mirror data can also be stored separately on the disk.

[0066] For a data page in the cache, if the operation logs corresponding to the data page from the previous checkpoint to the current checkpoint have all been flushed to disk. In other words, if the operations recorded in the data other than the page mirror data of the transaction logs that have been flushed to disk in the current disk flushing work already include all the operations performed on a data page from the previous checkpoint to the current checkpoint, then the data page can be flushed to disk in the current disk flushing work.

[0067] Then, obtain a data page to be flushed from the data pages in the cache that need to be flushed. When the data page to be flushed has page mirror data, obtain the corresponding page mirror data from the cache. Exemplarily, denote the data page to be flushed as page_cur, the table it belongs to in the disk as RelFileNode_cur, and the block number corresponding to its table as BlockNumber_cur. Refer to Figure 5 As shown, since the page mirror data records RelFileNode and BlockNumber, it is possible to compare RelFileNode_cur and BlockNumber_cur with the RelFileNode and BlockNumber recorded in the page mirror data to confirm whether there is page mirror data corresponding to the data page to be flushed in the global page data linked list.

[0068] Then, if there are data pages in other data pages that need to be flushed and belong to the same table as the data page to be flushed, and among the data pages that belong to the same table as the data page to be flushed, there are also data pages that have corresponding page mirror data in the cache. Generate a storage file in the disk and place these page mirror data in this storage file together.

[0069] If there are no data pages in other data pages that need to be flushed and belong to the same table as the data page to be flushed, or there are data pages that belong to the same table as the data page to be flushed but there are no data pages that have corresponding page mirror data in the cache among the data pages that belong to the same table as the data page to be flushed. Generate a storage file in the disk and place the page mirror data of the data page to be flushed in this storage file.

[0070] That is to say, a storage file in the disk may store one page mirror data or may store multiple page mirror data. Moreover, the file name of the storage file records the table location information in the disk of the table to which the data page corresponding to the page mirror data stored in the storage file belongs. Since even if there are multiple page mirror data in the storage file, the data pages corresponding to the multiple page mirror data must belong to the same table, only one table location information is required in the storage file.

[0071] In one embodiment, when generating a storage file, the file name of the storage file is generated using the table location information of the table to which the data page to be written to disk belongs, the block number of the data page to be written to disk in the table to which it belongs, and the log sequence number of the transaction log to which the page mirror data corresponding to the data page to be written to disk belongs. Exemplarily, the file name is RelFileNode_BlockNumber_lsn. In the file name, RelFileNode is the table location information of the table to which the data page to be written to disk belongs, BlockNumber is the block number of the data page to be written to disk in the table to which it belongs, and lsn is the log sequence number of the transaction log to which the page mirror data corresponding to the data page to be written to disk belongs. That is to say, regardless of whether the storage file needs to store one or more page mirror data, the file name of the storage file is generated using the table location information of the table to which the data page to be written to disk belongs, the block number of the data page to be written to disk in the table to which it belongs, and the log sequence number of the transaction log to which the page mirror data corresponding to the data page to be written to disk belongs.

[0072] As Figure 7 shown, in one embodiment, the method for processing a transaction log generally includes:

[0073] Step S101, obtain the table location information of the table to which the data page corresponding to the page mirror data to be searched belongs in the disk, the block number of the data page corresponding to the page mirror data to be searched in the table to which it belongs, and the log sequence number of the transaction log to which the page mirror data to be searched belongs, and record them as the target table location, the target block number, and the target sequence number respectively.

[0074] Combined with Figure 5 shown, in the content of each page mirror data, there is the table location information of the corresponding data page in the table in the disk, the block number of the corresponding data page in the table to which it belongs, and the log sequence number of the transaction log to which it belongs. When searching for a page mirror data, first obtain this information and record it as the target table location, the target block number, and the target sequence number.

[0075] Exemplarily, record the page mirror data to be searched as page_target, the target table location can be recorded as RelFileNode_target, the target block number can be recorded as BlockNumber_target, and the target sequence number can be recorded as lsn_target.

[0076] Step S102, traverse the storage files in the disk. Specifically, after determining the target table location, the target block number, and the target sequence number, traverse the storage files in the disk.

[0077] Step S103: Obtain the storage file in the disk whose table location information in the file name is the same as the target table location, and denote it as the preliminary target file. Since the table location information of the data page corresponding to the page mirror data stored in the storage file is recorded in the file name of the storage file, the table location information in the file name can be compared with the target table location. If they are the same, it indicates that the page mirror data to be searched may exist in this storage file.

[0078] That is to say, during the process of traversing the storage files, when a storage file whose table location information in the file name is the same as the target table location is detected, obtain this storage file and denote it as the preliminary target file.

[0079] Step S104: Obtain the page mirror data in the preliminary target file whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number. After obtaining the preliminary target file, further search for the page mirror data in the preliminary target file. If page mirror data whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number can be found, it indicates that the table location information of the data page corresponding to this page mirror data is consistent with the target table location information, the block number of the corresponding data page in the table to which it belongs is the same as the target block number, and the log sequence number of the transaction log to which it belongs is the same as the target sequence number. Then, determine this page mirror data as the target page mirror data to be searched. Just obtain this page mirror data.

[0080] In the solution of this embodiment, the page mirror data of the transaction log is stored in a storage file in the disk, and the table location information of the table to which the data page corresponding to the page mirror data stored in the storage file belongs is recorded in the file name of the storage file. On the one hand, the page mirror data of the transaction log and other data except the page mirror data are stored separately in the disk, so that when the database synchronizes the transaction log with other standby databases, only the other data except the page mirror data can be synchronized, thereby reducing the amount of data synchronization between the primary and standby databases, which helps to improve the synchronization efficiency. On the other hand, by obtaining the table position information of the table to which the data page corresponding to the page mirror data to be found belongs in the disk, the block number of the data page corresponding to the page mirror data to be found in the table to which it belongs, and the log sequence number of the transaction log to which the page mirror data to be found belongs, respectively recorded as the target table position, the target block number and the target sequence number, traversing the storage files in the disk; obtaining the storage file whose table position information in the file name in the disk is the same as the target table position, recorded as the preliminary target file; obtaining the page mirror data in the preliminary target file whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number, that is, when searching for page mirror data, only the storage file needs to be searched, and the page mirror data can be found more quickly than searching for the complete transaction log with page mirror data. Therefore, the convenience of using transaction logs is improved from multiple aspects, which helps to improve the operation efficiency of the database.

[0081] like Figure 8 As shown, in one embodiment, the transaction log processing method generally includes:

[0082] Step S201, obtain the table location information of the table to which the data page corresponding to the page mirror data to be found belongs in the disk, the block number of the data page corresponding to the page mirror data to be found in the table to which it belongs, and the log sequence number of the transaction log to which the page mirror data to be found belongs, and record them as the target table location, target block number and target sequence number respectively.

[0083] Step S202, traverse the storage files in the disk.

[0084] Step S203, obtaining a storage file in the disk whose table location information in the file name is the same as the target table location, and recording it as a preliminary target file.

[0085] Step S204, detect whether the preliminary target file stores only one page mirror data, if so, execute step S208, if not, execute step S205. Specifically, read the length of the first page mirror data in the preliminary target file (refer to Figure 5As shown, len records the length of the page mirror data), denoted as len_cur. Check whether the size of the preliminary target file (denoted as file_size) is equal to len_cur. If so, it means that the preliminary target file stores only one page mirror data. If not, it means that the preliminary target file stores multiple page mirror data.

[0086] Step S205: Read the page mirror data in the preliminary target file sequentially. Specifically, when there are multiple page mirror data in the preliminary target file, it is necessary to read the page mirror data in the preliminary target file sequentially.

[0087] Step S206: Detect whether the block number and log sequence number of the read page mirror data are the same as the target block number and target sequence number. If so, execute Step S209. If not, execute Step S207. For each read page mirror data, detect whether the block number and log sequence number of the page mirror data are the same as the target block number and target sequence number.

[0088] Step S207: Judge whether all the page mirror data in the preliminary target file have been read. If so, execute Step S210. If not, execute Step S205. Specifically, if one of the block number and log sequence number of the read page mirror data is not the same as the target block number and target sequence number, it means that the page mirror data is not the target page mirror data to be searched. Then, judge whether all the page mirror data in the preliminary target file have been read. If not, continue to read.

[0089] Exemplarily, denote the total length of the page mirror data that has been read in the preliminary target file as read_len. After reading the first page mirror data, read_len = len_cur. If the first page mirror data is not the target page mirror data, start reading the next page mirror data from read_len. If it is not the target page mirror data, read the length of the page mirror data, denoted as len_next, and update read_len = read_len + len_next.

[0090] When it is detected that a page mirror data is the target page mirror data, read the length of the page mirror data, denoted as len_next. Then, obtain the page mirror data between the current read_len and read_len + len_next, which is the target page mirror data.

[0091] If read_len is equal to the size of the preliminary target file (denoted as file_size), it means that all the page mirror data in the preliminary target file have been read and the target page mirror data has not been found.

[0092] Specifically, refer to Figure 9As shown, taking the preliminary target file storing three page mirror data as an example, it should be noted that the page mirror data also records its own length information, that is Figure 9 the content represented by len of Figure 9 . In addition, len1 represents the length of the first page mirror data, len2 represents the length of the second page mirror data, and len3 represents the length of the third page mirror data.

[0093] After reading the first page mirror data, read_len = len1. If the first page mirror data is not the target page mirror data, start reading the second page mirror data from read_len. If the second is not the target page mirror data, the length of the page mirror data read is len2, and update read_len = read_len + len2, that is, read_len is now len1 + len2.

[0094] Start reading the third page mirror data from read_len. If the third page mirror data is the target page mirror data, the length of the page mirror data read is len3, then obtain the page mirror data between the current read_len and read_len + len3, which is the target page mirror data.

[0095] If the third page mirror data is not the target page mirror data, the length of the page mirror data read is len3, and update read_len = read_len + len3, that is, read_len is now len1 + len2 + len3. Then read_len is equal to the size of the preliminary target file (denoted as file_size), which means that all the page mirror data in the preliminary target file have been read and the target page mirror data has not been found.

[0096] Step S208, detect whether the block number and log sequence number of a page mirror data in the preliminary target file are the same as the target block number and target sequence number. If so, execute step S209. If not, execute step S210. In the case where there is only one page mirror data in the preliminary target file, detect whether the block number and log sequence number corresponding to this one page mirror data are the same as the target block number and target sequence number.

[0097] Step S209, obtain the page mirror data in the preliminary target file whose block number is the same as the target block number and log sequence number is the same as the target sequence number.

[0098] Step S210, determine whether the traversal of the storage file on the disk is completed. If so, execute step S208. If not, return to execute step S202. Specifically, if the target page mirror data to be searched does not exist in the preliminary target file. Then, determine whether the traversal of the storage file is completed. If the traversal is not completed, continue the traversal.

[0099] Step S211 determines that there is no page mirror data to be searched for in the disk. Specifically, if the traversal of the stored files has been completed, then the page mirror data has not been found, and it is determined that there is no page mirror data to be searched for in the disk, and a prompt message can be output.

[0100] It should be noted that the steps not detailed in this embodiment refer to those described above.

[0101] As Figure 10 shown, in one embodiment, the file name of the stored file storing multiple page mirror data also records the maximum block number and the minimum block number of all page mirror data stored in the stored file, as well as the maximum log sequence number and the minimum log sequence number. Exemplarily, the file name format of the stored file is:

[0102] RelFileNode_minBlockNumber_minlsn.maxBlockNumber_maxlsn;

[0103] RelFileNode is the table location information of the table to which the data page corresponding to the page mirror data to be written to disk belongs, minBlockNumber is the minimum block number, minlsn is the minimum log sequence number, maxBlockNumber is the maximum block number, and maxlsn is the maximum log sequence number.

[0104] For the page mirror data storing only the corresponding data page of one data page, referring to the above, the file name is still generated as RelFileNode_BlockNumber_lsn.

[0105] In one embodiment, the processing method of the transaction log generally includes:

[0106] Step S301, obtain the table location information of the table to which the data page corresponding to the page mirror data to be searched for belongs in the disk, the block number of the data page corresponding to the page mirror data to be searched for in the table, and the log sequence number of the transaction log to which the page mirror data to be searched for belongs, and record them as the target table location, the target block number, and the target sequence number respectively.

[0107] Step S302, traverse the stored files in the disk.

[0108] Step S303, obtain the stored file in the disk whose table location information in the file name is the same as the target table location, and record it as the preliminary target file.

[0109] Step S304: Check whether the preliminary target file stores only one page mirror data. If so, execute Step S308; if not, execute Step S305. Specifically, it can be directly determined whether only one page mirror data is stored according to the file name format of the preliminary target file. That is, if the file name has a maximum block number and a minimum block number, as well as a maximum log sequence number and a minimum log sequence number, then there are multiple page mirror data.

[0110] Step S305: Determine whether the target block number is greater than or equal to the minimum block number and less than or equal to the maximum block number, and the target sequence number is greater than or equal to the minimum log sequence number and less than or equal to the maximum log sequence number. If so, execute Step S306; if not, execute Step S310. Specifically, if the target block number is greater than or equal to the minimum block number and less than or equal to the maximum block number, and the target sequence number is greater than or equal to the minimum log sequence number and less than or equal to the maximum log sequence number, it indicates that the target page mirror data may be in the preliminary target file, so the preliminary target file is retrieved. Conversely, it indicates that the target page mirror data is not in the preliminary target file, and there is no need to search for the page mirror data in the preliminary target file anymore.

[0111] Step S306: Conduct a further search in the preliminary target file. Specifically, it is to conduct a further search for the page mirror data in the preliminary target file.

[0112] In one implementation, the search can be performed by referring to the method of sequentially reading the page mirror data in the preliminary target file described above.

[0113] In addition, in one implementation, all the page mirror data in the storage file storing multiple page mirror data are arranged in ascending order of block number. The steps for further searching the preliminary target file include:

[0114] Obtain the differences between the target block number and the minimum block number, and between the maximum block number and the target block number; if the difference between the target block number and the minimum block number is greater than the difference between the maximum block number and the target block number, start the search from the last page mirror data in the preliminary target file; if the difference between the target block number and the minimum block number is less than or equal to the difference between the maximum block number and the target block number, start the search from the first page mirror data in the preliminary target file.

[0115] That is, when storing multiple page mirror data into the storage file, first arrange the multiple page mirror data in ascending order of block number, and then put them into the storage file. When searching, if the difference between the target block number and the minimum block number is greater than the difference between the maximum block number and the target block number, it indicates that the target block number is closer to the maximum block number, so start the search from the last page mirror data in the preliminary target file. If the difference between the target block number and the minimum block number is less than or equal to the difference between the maximum block number and the target block number, it indicates that the target block number is closer to the minimum block number, and start the search from the first page mirror data in the preliminary target file.

[0116] In addition, in one implementation, a storage file storing multiple page mirror data is configured with a file header, and the content of the file header includes the number of page mirror data in the storage file, a block number array, a log sequence number array, and a starting address array. Among them, the block numbers of all the page mirror data in the storage file are stored in the block number array, the log sequence numbers of all the page mirror data in the storage file are stored in the log sequence number array, and the starting addresses of all the page mirror data in the storage file are stored in the starting address array.

[0117] Refer to Figure 11 As shown, it is an example of a storage file with a file header. N in the file header is the number of page mirror data in the combined page mirror data. BlockNumber[N] represents the block number array, lsn[N] represents the log sequence number array, and ptr[N] represents the starting address array. In addition, the block numbers of each page mirror data in the block number array are arranged in the order of the page mirror data in the combined page mirror data, and the log sequence numbers of each page mirror data in the log sequence number array are arranged in the order of the page mirror data in the combined page mirror data.

[0118] Furthermore, as Figure 11 shown, there are three page mirror data in the combined page mirror data. The number of page mirror data in the combined page mirror data is 3. There are three block numbers in the block number array, three log sequence numbers in the log sequence number array, and three starting addresses in the starting address array. Refer to Figure 11 shown. Exemplarily, for example, if the three starting addresses are ptr[0], ptr[1], and ptr[2], then ptr[0] is the starting address of the first page mirror data, ptr[1] is the starting address of the second page mirror data, and ptr[2] is the starting address of the third page mirror data.

[0119] Refer to Figure 12 shown, the steps for further searching the preliminary target file include:

[0120] Step S401, read the file header. Specifically, that is to read the content of the file header of the preliminary target file. Specifically, this step includes calculating the length of the file header according to the header length calculation formula;

[0121] Among them, the header length calculation formula is:

[0122] header_len = sizeof(N) + (sizeof(BlockNumber[0]) + sizeof(lsn[0]) + sizeof(ptr[0])) * N.

[0123] The header_len is the length of the file header, sizeof(N) is the field length of the number of page mirror data records in the file header, BlockNumber[0] is the field length of the first block number in the block number array, lsn[0] is the field length of the first log sequence number in the log sequence number array, ptr[0] is the field length of the first starting address in the starting address array, and N is the number of page mirror data in the preliminary target file.

[0124] Specifically, since the field lengths of the log sequence number, block number, and starting address are the same, only the character content changes. Therefore, by adding the length of one block number, one log sequence number, and one starting address, the relevant content length of one page mirror data can be obtained. Then, multiplying it by the number of page mirror data can get the length of the same content of all page mirror data. Adding the field length of the record count in the file header can obtain the length of the entire file header. Then, the file header can be read according to the length of the file header.

[0125] Step S402, traverse the block number array and the log sequence number array.

[0126] Step S403, detect whether the block number and log sequence number at the same position are both the same as the target block number and target sequence number. If so, execute Step S404. If not, execute Step S405. Specifically, the block numbers of each page mirror data in the block number array are arranged in the order of the page mirror data in the combined page mirror data, and the log sequence numbers of each page mirror data in the log sequence number array are arranged in the order of the page mirror data in the combined page mirror data. The block number and log sequence number at the same position are the block number and log sequence number with the same order in their respective arrays.

[0127] Refer to Figure 11 As shown, there are three block numbers BlockNumber[0], BlockNumber[1], and BlockNumber[2] arranged in order in the block number array, and there are three log sequence numbers lsn[0], lsn[1], and lsn[2] arranged in order in the log sequence number array. BlockNumber[0] and lsn[0] are the block number and log sequence number at the same position, BlockNumber[1] and lsn[1] are the block number and log sequence number at the same position, and BlockNumber[2] and lsn[2] are the block number and log sequence number at the same position.

[0128] Step S404: Obtain the page mirror data in the preliminary target file where the block number is the same as the target block number and the log sequence number is the same as the target sequence number. If the block number and log sequence number at the same location are both the same as the target block number and the target sequence number, the page mirror data corresponding to the block number and log sequence number at the same location is the target page mirror data. At this time, the page mirror data can be found according to the corresponding starting address in the file header, and then the page mirror data is read according to the length of the page mirror data, that is, the page mirror data with the block number the same as the target block number and the log sequence number the same as the target sequence number is obtained.

[0129] Exemplarily, if BlockNumber[1] and lsn[1] are both the same as the target block number and the target sequence number, then obtain ptr[1], find the page mirror data according to ptr[1], and then read the length of the page mirror data from the page mirror data (i.e., Figure 11 the len), and read the page mirror data.

[0130] Step S405: Determine that the page mirror data to be searched does not exist in the preliminary target file.

[0131] Step S307: Check whether the page mirror data to be searched is found in the preliminary target file. If so, execute Step S309; if not, execute Step S310. Specifically, that is, after further searching the preliminary target file, whether the target page mirror data is found.

[0132] Step S308: Detect whether a page mirror data in the preliminary target file has a block number the same as the target block number and a log sequence number the same as the target sequence number. If so, execute Step S309; if not, execute Step S310. In the case where there is only one page mirror data in the preliminary target file, detect whether the block number and log sequence number corresponding to this one page mirror data are the same as the target block number and the target sequence number.

[0133] Step S309: Obtain the page mirror data in the preliminary target file where the block number is the same as the target block number and the log sequence number is the same as the target sequence number.

[0134] Step S310: Check whether the traversal of the storage file on the disk is completed. If so, execute Step S311; if not, return to execute Step S302. Specifically, if the target page mirror data to be searched does not exist in the preliminary target file. Then, check whether the traversal of the storage file is completed. If the traversal is not completed, continue the traversal.

[0135] Step S311: Determine that the page mirror data to be searched does not exist on the disk. Specifically, if the traversal of the storage file has been completed, then the page mirror data has not been found, and it is determined that the page mirror data to be searched does not exist on the disk, and a prompt message can be output.

[0136] In the solution of this embodiment, when multiple page mirror data are stored in the preliminary target file, it is determined whether the target block number is greater than or equal to the minimum block number and less than or equal to the maximum block number, and the target sequence number is greater than or equal to the minimum log sequence number and less than or equal to the maximum log sequence number. If so, the preliminary target file is further searched. That is to say, the target block number and the target sequence number are directly compared with the minimum block number, the maximum block number, the minimum log sequence number, and the maximum log sequence number in the preliminary target file name, so as to determine whether there may be page mirror data to be searched in the preliminary target file. If there may be, the preliminary target file is further searched. If not, there is no need to search the preliminary target file anymore, making the search faster.

[0137] In addition, by reading the file header and traversing the block number array and the log sequence number array, it is detected whether there are block numbers and log sequence numbers at the same position that are both the same as the target block number and the target sequence number. If so, the page mirror data in the preliminary target file with the same block number as the target block number and the same log sequence number as the target sequence number is obtained. That is to say, when there may be page mirror data to be searched in the preliminary target file, only the data in the file header is used to complete the search of the preliminary target file, reducing the number of data reads and improving the search efficiency.

[0138] This embodiment also provides a machine-readable storage medium and a computer device. Figure 13 It is a schematic diagram of a machine-readable storage medium 10 according to an embodiment of the present invention. Figure 14 It is a schematic diagram of a computer device 20 according to an embodiment of the present invention.

[0139] The machine-readable storage medium 10 stores a machine-executable program 11 thereon. When the machine-executable program 11 is executed by a processor, the transaction log processing method of the database in any of the above embodiments is implemented.

[0140] The computer device 20 may include a memory 210, a processor 220, and a machine-executable program 11 stored in the memory 210 and running on the processor 220. When the processor 220 executes the machine-executable program 11, the transaction log processing method of the database in any of the above embodiments is implemented.

[0141] For the description of this embodiment, the machine-readable storage medium 10 can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the machine-readable storage medium 10 can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0142] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0143] The computer device 20 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smart phone. In some examples, the computer device 20 can be a cloud computing node. The computer device 20 can be described in the general context of computer system-executable instructions, such as program modules, executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer device 20 can be implemented in a distributed cloud computing environment where tasks are executed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0144] The computer device 20 can include a processor 220 adapted to execute stored instructions and a memory 210 that provides temporary storage space for the operation of the instructions during operation. The processor 220 can be a single-core processor, a multi-core processor, a computing cluster, or any other number of other configurations. The memory 210 can include a random access memory (RAM), a read-only memory, a flash memory, or any other suitable storage system.

[0145] The processor 220 can be connected to an I / O interface (input / output interface) suitable for connecting the computer device 20 to one or more I / O devices (input / output devices) through a system interconnection such as PCI, PCI-Express, etc. The I / O devices can include, for example, a keyboard and a pointing device, where the pointing device can include a touchpad or a touch screen, etc. The I / O devices can be built-in components of the computer device 20 or can be devices externally connected to the computing device.

[0146] The processor 220 can also be linked to a display interface suitable for connecting the computer device 20 to a display device through a system interconnection. The display device can include a display screen as a built-in component of the computer device 20. The display device can also include a computer monitor, a television set, a projector, etc. externally connected to the computer device 20. In addition, a network interface controller (NIC) can be suitable for connecting the computer device 20 to a network through a system interconnection. In some embodiments, the NIC can use any suitable interface or protocol (such as Internet Small Computer System Interface, etc.) to transmit data. The network can be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, etc. Remote devices can be connected to the computing device through the network.

[0147] The solution of the present invention is applicable to relational databases, and particularly applicable to the KingbaseES database (referred to as the KES database), enriching the functions of the database and improving the efficiency of the database.

[0148] At this point, those skilled in the art should recognize that although multiple exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived from the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and determined to cover all these other variations or modifications.

Claims

1. A method for processing transaction logs of a database, wherein, The page mirror data of the transaction log is stored in a storage file configured on the disk, and the storage file stores page mirror data corresponding to at least one data page belonging to the same table. The file name of the storage file records the table location information of the table to which the data page corresponding to the page mirror data stored in the storage file belongs on the disk. Moreover, the transaction log processing method of the database includes: Obtain the table location information of the table to which the data page corresponding to the page mirror data to be searched belongs on the disk, the block number of the data page corresponding to the page mirror data to be searched in the table to which it belongs, and the log sequence number of the transaction log to which the page mirror data to be searched belongs, and record them as the target table location, the target block number, and the target sequence number respectively; Traverse the storage files on the disk; Obtain the storage file on the disk whose table location information in the file name is the same as the target table location, and record it as the preliminary target file; Obtain the page mirror data in the preliminary target file whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number.

2. The method for processing a transaction log of a database according to claim 1, wherein, Before the step of obtaining the page mirror data in the preliminary target file whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number, it includes: Detect whether the preliminary target file stores only one page mirror data; If so, detect whether one page mirror data in the preliminary target file has the same block number as the target block number and the same log sequence number as the target sequence number. If so, execute the step of obtaining the page mirror data in the preliminary target file whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number.

3. The method for processing the transaction log of a database according to claim 2, wherein, After the step of detecting whether one page mirror data in the preliminary target file has the same block number as the target block number and the same log sequence number as the target sequence number, it includes: If not, judge whether the traversal of the storage files on the disk is completed. If so, determine that the page mirror data to be searched does not exist on the disk. If not, continue to execute the step of traversing the storage files on the disk.

4. The method for processing a transaction log of a database according to claim 3, wherein, After the step of detecting whether the preliminary target file stores only one page mirror data, it includes: If not, sequentially read the page mirror data in the preliminary target file, and detect whether the read page mirror data has the same block number as the target block number and the same log sequence number as the target sequence number. If so, execute the step of obtaining the page mirror data in the preliminary target file whose block number is the same as the target block number and whose log sequence number is the same as the target sequence number; If not, judge whether all the page mirror data in the preliminary target file have been read. If so, execute the step of judging whether the traversal of the storage files on the disk is completed. If not, continue to execute the step of sequentially reading the page mirror data in the storage file.

5. The method for processing a transaction log of a database according to claim 3, wherein, The file name of the storage file storing multiple page mirror data also records the maximum block number and the minimum block number of all the page mirror data stored in the storage file, as well as the maximum log sequence number and the minimum log sequence number; After the step of detecting whether the preliminary target file stores only one page mirror data, it includes: If not, determine whether the target block number is greater than or equal to the minimum block number and less than or equal to the maximum block number, and whether the target sequence number is greater than or equal to the minimum log sequence number and less than or equal to the maximum log sequence number. If so, further search the preliminary target file. If not, execute the step of determining whether all the storage files on the disk have been traversed.

6. The method for processing a transaction log of a database according to claim 5, wherein, All the page mirror data in the storage file storing multiple page mirror data are arranged in ascending order of block numbers. The step of further searching the preliminary target file includes: Obtain the differences between the target block number and the minimum block number, and between the maximum block number and the target block number. If the difference between the target block number and the minimum block number is greater than the difference between the maximum block number and the target block number, start searching from the last page mirror data in the preliminary target file. If the difference between the target block number and the minimum block number is less than or equal to the difference between the maximum block number and the target block number, start searching from the first page mirror data in the preliminary target file.

7. The method for processing a transaction log of a database according to claim 5, wherein, The storage file storing multiple page mirror data is configured with a file header, and the content of the file header includes the number of page mirror data in the storage file, the block number array, the log sequence number array, and the first address array. Among them, the block number array stores the block numbers of all the page mirror data in the storage file, the log sequence number array stores the log sequence numbers of all the page mirror data in the storage file, and the first address array stores the first addresses of all the page mirror data in the storage file. The step of further searching the preliminary target file includes: Read the file header. Traverse the block number array and the log sequence number array. Detect whether there are block numbers and log sequence numbers at the same position that are both the same as the target block number and the target sequence number. If so, execute the step of obtaining the page mirror data in the preliminary target file with the block number the same as the target block number and the log sequence number the same as the target sequence number. If not, execute the step of determining whether all the storage files on the disk have been traversed.

8. The method for processing a transaction log of a database according to claim 7, wherein, The step of reading the file header includes: Calculate the length of the file header according to the header length calculation formula. Among them, the header length calculation formula is: header_len = sizeof(N)+(sizeof(BlockNumber[0])+sizeof(lsn[0])+sizeof(ptr[0]))*N; header_len is the length of the file header, sizeof(N) is the field length of the field recording the number of page mirror data in the file header, BlockNumber[0] is the field length of the first block number in the block number array, lsn[0] is the field length of the first log sequence number in the log sequence number array, ptr[0] is the field length of the first first address in the first address array, and N is the number of page mirror data in the preliminary target file. Read the file header according to the length of the file header.

9. A machine-readable storage medium having a machine-executable program stored thereon, and when the machine-executable program is executed by a processor, a method for processing a transaction log of a database according to any one of claims 1 to 8 is implemented.

10. A computer device, comprising a memory, a processor, and a machine-executable program stored on the memory and running on the processor, and when the processor executes the machine-executable program, a method for processing a transaction log of a database according to any one of claims 1 to 8 is implemented.

Images