Database transaction log processing method, storage medium and equipment

Abstract

The invention relates to a database technology, in particular to a processing method of a transaction log of a database, a storage medium and equipment. The method for processing the transaction logs of the database comprises the steps of obtaining range information of the transaction logs generated in the process that the database executes basic backup operation; according to the range information, page mirror image data generated by the database in the process of executing the basic backup operation is obtained, and the page mirror image data is data of a data page recorded in the transaction log; and storing page mirror image data generated by the database in the process of executing the basic backup operation into the backup set. According to the processing method for the transaction log of the database, the page mirror image data in the transaction log generated during the basic backup period is stored in the backup set, the page mirror image data required by page repair are accurately reserved while the safety of the basic backup data is ensured, the problem of page repair failure caused by improper storage of the page mirror image data is avoided, and the page repair efficiency is improved. And the overall performance of the database is effectively improved.

Description

Technical Field

[0001] The present invention relates to database technology, and particularly to a method for processing transaction logs of a database, a storage medium, and a device. Background Art

[0002] When a database performs a base backup, if the database is running business and a database page is copied into the backup set during the process of writing data, page breaks will occur in the pages in the base backup. To repair the possible page break problems during the base backup, the existing solution is to retain the XLOG logs carrying page data, so that when page breaks are found, the page data in the XLOG logs can be used for replacement and repair.

[0003] However, currently, database clusters usually adopt the synchronous streaming replication mode to keep the data of the primary and standby databases strictly consistent. However, when the primary database is under heavy business pressure, a large amount of XLOG logs will be generated. At this time, the synchronous streaming replication mode is likely to become a performance bottleneck of the database. In such a case, to reduce the data volume of the XLOG logs, what those skilled in the art may think of is: separating the page data in the XLOG logs of the database, continuously storing it in a page data file, and then separately writing the page data in the XLOG logs and the XLOG logs without page data to disk.

[0004] Although separating the page data in the XLOG logs from the XLOG logs can reduce the data volume of the XLOG logs, the storage method of the XLOG logs and their page data files generated during the base backup still needs to be improved.

[0005] The above information disclosed in this background art is only used to increase the understanding of the background art of the present application. Therefore, it may include prior art that is not known to those of ordinary skill in the art. Summary of the Invention

[0006] In view of the above problems, a method for processing transaction logs of a database, a storage medium, and a device that overcome the above problems or at least partially solve the above problems are proposed.

[0007] An object of the present invention is to provide a storage solution for transaction logs and their page mirror data generated during a base backup, so as to accurately retain the page mirror data required for page repair while ensuring the security of the base backup data.

[0008] A further object of the present invention is to improve the accuracy of obtaining the page mirror data generated during the base backup.

[0009] In particular, the present invention provides a method for processing a transaction log of a database, which includes:

[0010] Obtaining range information of the transaction log generated during the execution of the base backup operation of the database;

[0011] According to the range information, obtaining page mirror data generated during the execution of the base backup operation of the database, where the page mirror data is the data of the data pages recorded in the transaction log;

[0012] Storing the page mirror data generated during the execution of the base backup operation of the database into the backup set.

[0013] Optionally, before the step of obtaining, according to the range information, the page mirror data generated during the execution of the base backup operation of the database, the method for processing the transaction log of the database further includes:

[0014] Separating the page mirror data and the transaction log in the transaction log generated during the execution of the base backup operation of the database to obtain the transaction log after the separation of the page mirror data;

[0015] Storing the page mirror data separated from the transaction log into a page mirror file, where the page mirror file is used to store the page mirror data separated from the transaction log of the database.

[0016] Optionally, the page mirror file includes a used interval and a to-be-written interval. The used interval is used to store the page mirror data in the transaction log generated before the execution of the base backup operation of the database; and

[0017] Storing the page mirror data separated from the transaction log into the page mirror file includes:

[0018] Obtaining the to-be-written interval of the page mirror file;

[0019] Storing the page mirror data in the transaction log generated during the execution of the base backup operation of the database into the to-be-written interval of the page mirror file.

[0020] Optionally, after the step of obtaining, according to the range information, the page mirror data generated during the execution of the base backup operation of the database, the method for processing the transaction log of the database further includes:

[0021] Storing the transaction log after the separation of the page mirror data generated during the execution of the base backup operation of the database into the backup set.

[0022] Optionally, obtaining, according to the range information, the page mirror data generated during the execution of the base backup operation of the database includes:

[0023] According to the range information, obtaining the transaction log generated during the execution of the base backup operation of the database;

[0024] Obtain the page mirror data retained in the transaction log generated during the execution of the base backup operation on the database.

[0025] Optionally, storing the page mirror data generated during the execution of the base backup operation on the database in the backup set includes:

[0026] Store the transaction log generated during the execution of the base backup operation on the database in the backup set, and the transaction log stored in the backup set contains the page mirror data.

[0027] Optionally, after the step of storing the page mirror data generated during the execution of the base backup operation on the database in the backup set, the method for processing the transaction log of the database further includes:

[0028] After the base backup operation ends, create a page mirror file, which is used to store the page mirror data in the transaction log generated after the base backup operation of the database;

[0029] Separate the page mirror data and the transaction log in the transaction log generated by the database, and store them in the page mirror file.

[0030] Optionally, obtaining the range information of the transaction log generated during the execution of the base backup operation on the database includes:

[0031] Obtain the position of the redo point recorded in the first checkpoint log created before the start of the base backup operation;

[0032] Obtain the end position of the second checkpoint log created after the end of the base backup operation;

[0033] Take the range between the position of the redo point and the end position as the range information of the transaction log generated during the execution of the base backup operation on the database.

[0034] According to another aspect of the present invention, there is also provided a machine-readable storage medium, on which a machine-executable program is stored, and when the machine-executable program is executed by a processor, it implements the method for processing the transaction log of the database as described in any one of the above.

[0035] According to still another aspect of the present invention, there is also provided a computer device, including a memory, a processor, and a machine-executable program stored on the memory and running on the processor, and when the processor executes the machine-executable program, it implements the method for processing the transaction log of the database as described in any one of the above.

[0036] The method for processing the transaction log of the database of the present invention realizes the stable storage of page mirror data during base backup by obtaining the range information of the transaction log generated during the execution of the base backup operation of the database, obtaining the page mirror data generated during the execution of the base backup operation of the database according to the range information, and storing the page mirror data generated during the execution of the base backup operation of the database into the backup set. The method for processing the transaction log of the database of the present invention stores the page mirror data in the transaction log generated during base backup into the backup set, ensuring the security of the base backup data while precisely retaining the page mirror data required for page repair, and improving the stability of the database.

[0037] Furthermore, the method for processing the transaction log of the database of the present invention stores the transaction log generated during the execution of the base backup operation of the database into the backup set, realizing the stable storage of the transaction log generated during base backup.

[0038] Even further, the method for processing the transaction log of the database of the present invention realizes the accurate acquisition of the range of the transaction log generated during base backup by obtaining the redo point recorded in the first checkpoint log created before the start of the base backup operation and the end position of the second checkpoint log created after the end of the base backup operation, and taking the range between the redo point and the end position as the range information of the transaction log generated during the execution of the base backup operation of the database, improving the accuracy of obtaining the page mirror data generated during base backup.

[0039] Based on the following detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more clear about the above and other purposes, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Some specific embodiments of the present invention will be described in detail hereinafter with reference to the accompanying drawings in an exemplary but non-limiting manner. The same reference numerals in the drawings denote the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0041] Figure 1 is a schematic flowchart of the method for processing the transaction log of the database according to an embodiment of the present invention;

[0042] Figure 2 is a schematic structural diagram of the transaction log in the method for processing the transaction log of the database according to an embodiment of the present invention;

[0043] Figure 3 is a schematic structural diagram of the transaction log generated during the execution of the base backup operation in the method for processing the transaction log of the database according to an embodiment of the present invention;

[0044] Figure 4 Schematic diagram of the structure of a page mirror file in a method for processing a transaction log of a database according to an embodiment of the present invention;

[0045] Figure 5 Schematic diagram of the structure of a page mirror file in a method for processing a transaction log of a database according to another embodiment of the present invention;

[0046] Figure 6 Schematic diagram of the structure of a transaction log generated during the execution of a base backup operation in a method for processing a transaction log of a database according to another embodiment of the present invention;

[0047] Figure 7 Schematic diagram of the structure of a page mirror file in a method for processing a transaction log of a database according to still another embodiment of the present invention;

[0048] Figure 8 Schematic flow diagram of a method for processing a transaction log of a database according to an embodiment of the present invention;

[0049] Figure 9 Schematic flow diagram of a method for processing a transaction log of a database according to another embodiment of the present invention;

[0050] Figure 10 Schematic diagram of a machine-readable storage medium according to an embodiment of the present invention; and

[0051] Figure 11 Schematic diagram of a computer device according to an embodiment of the present invention. Detailed implementation manners

[0052] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that this disclosure can be more thoroughly understood and the scope of the present invention can be fully conveyed to those skilled in the art.

[0053] To solve the above technical problems, an embodiment of the present invention proposes a method for processing a transaction log of a database. Figure 1 Schematic flow chart of a method for processing a transaction log of a database according to an embodiment of the present invention. Figure 2 Schematic diagram of the structure of a transaction log in a method for processing a transaction log of a database according to an embodiment of the present invention. As Figure 1 shown, the method for processing a transaction log of a database generally may include:

[0054] Step S102: Obtain the range information of the transaction log generated during the execution of the base backup operation on the database.

[0055] Step S104: According to the range information, obtain the page mirror data generated during the execution of the base backup operation on the database. The page mirror data is the data of the data pages recorded in the transaction log.

[0056] Step S106: Store the page mirror data generated during the execution of the base backup operation on the database into the backup set.

[0057] In a database, the transaction log refers to the XLOG log, and the XLOG log details the operation process of the service process on the database. During the operation of the database, multiple XLOG logs will be generated. As Figure 2 shown, an XLOG log can include an XLOG log header (denoted as head data) and an XLOG log data area. Specifically, the XLOG log data area can include multiple block data areas (denoted as block) and main data (denoted as main data). Each block can contain page mirror data (denoted as page data) and tuple data (denoted as tupledata). The XLOG log header includes the XLogRecord structure, the header data of each block, and the header data of the main data.

[0058] It should be noted that different types of XLOG logs have different compositions. Each XLOG log contains head data, but not necessarily page data, tupledata, and main data. Some types of XLOG logs only have head data without an XLOG log data area; some other types of XLOG logs only have head data and main data; and some other types of XLOG logs only have head data, page data, and tupledata, and each block may contain both page data and tupledata, or may only contain tupledata. That is to say, an XLOG log may or may not contain page data.

[0059] During the operating system crash of the database, some operating system pages (such as pages with a size of 4KB) may not have been written to the disk in time, which may cause a database page (such as a page composed of two operating system pages) to contain a mixture of old and new data. During the recovery after the operating system crash of the database, since the information stored in the XLOG log is not complete enough, it is impossible to fully recover the database pages that contain a mixture of old and new data in the database. In the prior art, in order to recover such database pages, after the redo point (denoted as the redo point) of the checkpoint (checkpoint log) in the database, when the database page (with a default size of 8KB) is modified (updated) for the first time, the entire database page is written to the XLOG log. At this time, the XLOG log contains page mirror data to save the complete database page mirror. It should be noted that the redo point is a special point in the XLOG log. Before this point, all the data in the database is the same as the information reflected by the XLOG log that has been written to the disk. During the creation of the checkpoint, a special XLOG log (denoted as the checkpoint log) will be created. A redo point will be recorded in this checkpoint log. Only when the XLOG log and user data before this redo point have been written to the disk, the checkpoint can be successfully created. When the database crashes, we can find a redo point from the nearest checkpoint log, and then start playing back the XLOG log from this redo point to recover the database. In the case of a broken page during the recovery of the database, the broken page is replaced according to the page mirror data in the XLOG log, so as to ensure the security and stability of the database data.

[0060] The method for processing the transaction log of the database in this embodiment realizes the stable storage of page mirror data during the base backup by obtaining the range information of the XLOG log generated during the execution of the base backup operation of the database, obtaining the page mirror data generated during the execution of the base backup operation of the database according to the range information, and storing the page mirror data generated during the execution of the base backup operation of the database in the backup set. The method for processing the transaction log of the database in this embodiment stores the page mirror data in the XLOG log generated during the base backup in the backup set, which not only ensures the security of the base backup data, but also accurately retains the page mirror data required for page repair, improving the stability of the database.

[0061] Since the page mirror data in the XLOG log is the data of the database page, the data volume of the XLOG log containing the page mirror data is large. In some embodiments, in order to reduce the data volume of the XLOG log, before the above step S104, the method for processing the transaction log of the database of the present invention may further include the following steps: separating the page mirror data in the XLOG log generated during the basic backup operation of the database from the XLOG log to obtain the XLOG log after the page mirror data is separated; storing the page mirror data separated from the XLOG log into a page mirror file, where the page mirror file is used to store the page mirror data separated from the XLOG log of the database.

[0062] In this embodiment, separating the page mirror data in all the XLOG logs generated in the database greatly reduces the data volume of the XLOG log, so that after the primary and standby database clusters are built, the host only transmits the XLOG log after the page mirror data is separated and the XLOG log without page mirror data to the standby machine, thereby greatly reducing the pressure of XLOG log transmission in the streaming replication between the host and the standby machine on the premise of ensuring the security and stability of the database data.

[0063] Figure 3 It is a schematic structural diagram of the transaction log generated during the basic backup operation in the method for processing the transaction log of the database according to an embodiment of the present invention. As Figure 3 shown, the page mirror data in the XLOG log generated during the basic backup period is separated from the XLOG log and stored separately in a page mirror file (denoted as the page data file).

[0064] In this embodiment, the above step S102 may include the following steps: obtaining the position of the redo point recorded in the first checkpoint log created before the basic backup operation starts; obtaining the end position of the second checkpoint log created after the basic backup operation ends; using the range between the position of the redo point and the end position as the range information of the XLOG log generated during the basic backup operation of the database.

[0065] Specifically, as Figure 3As shown, in the database, a checkpoint log (denoted as the checkpoint log) is created before and after the creation of the base backup, serving as the boundary between the start and end of the base backup. The start position information of the two checkpoint logs is recorded in a specified file (backup_label). The start position of the first checkpoint log is denoted as backup_begin_lsn, and the start position of the second checkpoint log is denoted as backup_end_lsn. The position of the redo point (denoted as the redo point) recorded in the first checkpoint log can be denoted as begin_redo_lsn. The redo point recorded in the second checkpoint log is denoted as end_redo_lsn. The end position of the second checkpoint log is denoted as the end lsn.

[0066] On this basis, the range of XLOG logs generated during the base backup is denoted as: from begin_redo_lsn to the end lsn of the checkpoint log at backup_end_lsn. That is to say, the range information of the XLOG logs generated by the database during the execution of the base backup operation is from begin_redo_lsn to the end lsn of the checkpoint log at backup_end_lsn.

[0067] The method for processing the transaction log of the database in this embodiment realizes the accurate acquisition of the range of XLOG logs generated during the base backup by obtaining the redo point recorded in the first checkpoint log created before the start of the base backup operation and the end position of the second checkpoint log created after the end of the base backup operation, and taking the range between the redo point and the end position as the range information of the XLOG logs generated by the database during the execution of the base backup operation, improving the accuracy of obtaining the page mirror data generated during the base backup.

[0068] Figure 4 It is a schematic diagram of the structure of the page mirror file in the method for processing the transaction log of the database according to an embodiment of the present invention. As Figure 4 shown, the page mirror file includes multiple page mirror data areas. The multiple page mirror data areas are set in one-to-one correspondence with the page mirror data stored in the page mirror file.

[0069] Each page mirror data area includes page mirror data and related data of the page mirror data. The related data of the page mirror data includes the start position of the XLOG log where the page mirror data is located (denoted as lsn), the page mirror data header (denoted as pagedata header), the position information of the data page corresponding to the page mirror data, and the total length of the page mirror data area (denoted as len).

[0070] The LSNs corresponding to each page mirror data increase sequentially. The LSN can be used to determine the correspondence between the page mirror data and the XLOG. A data page refers to a page on the data disk. The location information of the data page corresponding to the page mirror data includes RelFileNode and BlockNumber. RelFileNode is the location information of the table where the data disk page corresponding to the page mirror data is located, and BlockNumber is the block number of the data disk page corresponding to the page mirror data in its table. RelFileNode and BlockNumber together determine the location of the data disk page on the disk. Based on these two data, RelFileNode and BlockNumber, a unique data disk page can be determined on the disk. When searching on the disk, it can be determined whether a version of the target page mirror data is found based on these two values. The page mirror data header is the header information corresponding to the page mirror data. len is used to calculate the starting address of the next page mirror data area in the page mirror file. When traversing the page mirror file, the next page mirror data area can be quickly found based on the len of each page mirror data area.

[0071] Figure 5 It is a schematic diagram of the structure of a page mirror file in the method for processing transaction logs of a database according to an embodiment of the present invention. As Figure 5 shown, the page mirror file includes a used interval and a to-be-written interval.

[0072] The used interval is used to store the page mirror data in the XLOG generated by the database before performing a base backup operation, and the to-be-written interval is used to store the page mirror data in the XLOG generated by the database during the base backup operation.

[0073] On this basis, the step of storing the page mirror data separated from the XLOG into the page mirror file may include the following steps: obtaining the to-be-written interval of the page mirror file; storing the page mirror data in the XLOG generated by the database during the base backup operation into the to-be-written interval of the page mirror file. Thus, the page mirror data corresponding to the XLOG generated during the base backup is stored in the page mirror file without discrimination.

[0074] When a base backup exists, the page mirror data generated during the base backup is not allowed to be cleared. If a page mirror file contains both page mirror data generated during the base backup and page mirror data generated during non-base backup periods, then this page mirror file is retained.

[0075] After the primary and standby database clusters are built, if a page break is found during standby replay, the corresponding page mirror data is requested from the primary machine, and the broken database page is replaced according to the page mirror data, thereby ensuring the consistency of the data between the standby machine and the primary machine while ensuring the efficiency of streaming replication.

[0076] In some embodiments, the above step S104 may include the following steps: obtaining the corresponding page mirror data from the page mirror file according to the range information. The above step S106 may include the following steps: storing the obtained corresponding page mirror data in the backup set

[0077] Further, after the above step S104, the method for processing the transaction log of the database of the present invention may further include the following steps: storing the XLOG log after separating the page mirror data in the XLOG log generated during the basic backup operation of the database in the backup set.

[0078] The method for processing the transaction log of the database in this embodiment stores the XLOG log and page mirror data generated during the basic backup operation of the database in the backup set, realizing the stable storage of the XLOG log and page mirror data generated during the basic backup.

[0079] In some embodiments, the step of separating the page mirror data and the XLOG log in the XLOG log generated during the basic backup operation of the database to obtain the XLOG log after separating the page mirror data may include the following steps: during the stage of assembling the XLOG log, separating the page mirror data in the XLOG log to be assembled from the XLOG log to be assembled to obtain the XLOG log after separating the page mirror data.

[0080] Further, after the step of obtaining the XLOG log after separating the page mirror data, the method for processing the transaction log of the database of the present invention may further include the following steps: pre-allocating a storage location for the XLOG log after separating the page mirror data in the XLOG log write cache according to the length of the XLOG log after separating the page mirror data; copying the XLOG log after separating the page mirror data to the XLOG log write cache according to the pre-allocated storage location for the XLOG log after separating the page mirror data.

[0081] The method for processing the transaction log of the database in this embodiment selects to separate the page mirror data in the XLOG log to be assembled containing page mirror data from the XLOG log to be assembled during the assembly stage of the XLOG log, and determines the storage location information of the XLOG log after it is copied to the XLOG log write cache according to the data length of the XLOG log after the page mirror data, avoiding the change of the storage location information caused by the change of the length of the XLOG log and maintaining the stability of the storage location information of the XLOG log in the XLOG log write cache.

[0082] Further, the step of separating the page mirror data in the XLOG log to be assembled from the XLOG log to be assembled may include the following steps: creating a page mirror linked list node for each page mirror data in the XLOG log to be assembled, and sequentially connecting the page mirror linked list nodes of all the page mirror data in the XLOG log to be assembled to form a page mirror linked list; connecting the page mirror linked list to a globally created global page data linked list; and sequentially connecting the remaining nodes in the first assembly linked list except the assembly linked list nodes for storing page mirror data to form a second assembly linked list for storing the XLOG log after the page mirror data is separated.

[0083] In this embodiment, the globally created global page data linked list includes multiple connected page mirror linked lists for storing all the page mirror data separated from the XLOG log in the database. Each page mirror linked list contains at least one page mirror linked list node. Each page mirror linked list node corresponds to a page mirror data, and each page mirror linked list node includes a page mirror data area and a pointer identifier (denoted as next). Next is used to point to the next page mirror linked list node to connect the page mirror linked lists corresponding to two adjacent XLOG logs.

[0084] The method for processing the transaction log of the database in this embodiment completes the operation of separating the page mirror data of the XLOG log to be assembled by creating a page mirror linked list node for each page mirror data in the XLOG log to be assembled, sequentially connecting the page mirror linked list nodes of all the page mirror data in the XLOG log to be assembled to form a page mirror linked list, connecting the page mirror linked list to a globally created global page data linked list, and sequentially connecting the remaining nodes in the first assembly linked list except the assembly linked list nodes for storing page mirror data to form a second assembly linked list, further improving the accuracy of the operation of separating the page mirror data from the XLOG log.

[0085] In a specific embodiment, after the step of sequentially connecting the page mirror linked list nodes of all the page mirror data in the XLOG log to be assembled to form a page mirror linked list, the method for processing the transaction log of the database of the present invention may further include the following steps: connecting the page mirror linked list after the remaining nodes in the first assembly linked list except for the assembly linked list nodes used to store the page mirror data.

[0086] That is to say, after sequentially connecting the page mirror linked list nodes of all the page mirror data in the XLOG log to be assembled to form a page mirror linked list, first connect the page mirror linked list to the end of the first assembly linked list, and then move out the page mirror linked list behind the first assembly linked list and connect it to the global page data linked list.

[0087] The method for processing the transaction log of the database in this embodiment can orderly adjust the nodes of the page mirror data, so that the data can be effectively managed and maintained globally, which is beneficial to data sharing and consistency maintenance, and improves the stability of the database.

[0088] In some embodiments, the step of storing the page mirror data in the XLOG log generated during the execution of the base backup operation of the database into the to-be-written interval of the page mirror file may include the following steps: moving out the page mirror linked list nodes with lsn within the range information from the global page data linked list; sequentially storing the page mirror linked list nodes moved out of the global page data linked list into a temporary linked list according to the ascending order of lsn in the page mirror linked list nodes; sequentially obtaining all the page mirror data areas from the temporary linked list according to the len order, and connecting the sequentially obtained page mirror data areas to form a continuous data area; sequentially writing the data of the continuous data area to the position adjacent to the used interval in the to-be-written interval of the page mirror file.

[0089] Specifically, the page mirror linked list nodes moved out of the global page data linked list can be sequentially stored in the temporary linked list according to the ascending order of lsn in the page mirror linked list nodes.

[0090] In addition, the to-be-written interval is physically located after the used interval in the actual physical storage space. That is to say, the step of sequentially writing the data of the continuous data area to the position adjacent to the used interval in the to-be-written interval of the page mirror file can be specifically executed as: sequentially writing the data of the continuous data area after the used interval in the page mirror file.

[0091] Figure 6 It is a schematic diagram of the structure of the transaction log generated during the execution of the base backup operation in the method for processing the transaction log of the database according to another embodiment of the present invention. As Figure 6 shown, the page mirror data in the XLOG log generated during the base backup period remains in the XLOG log.

[0092] In this embodiment, step S102 may include the following steps: obtaining the position of the redo point recorded in the first checkpoint log created before the start of the base backup operation; obtaining the end position of the second checkpoint log created after the end of the base backup operation; using the range between the position of the redo point and the end position as the range information of the XLOG log generated by the database during the execution of the base backup operation.

[0093] Specifically, as Figure 6 shown, in the database, before and after the creation of the base backup, a checkpoint log (denoted as the checkpoint log) will be created as the boundary between the start and end of the base backup. The start position information of both checkpoint logs is recorded in a specified file (backup_label). The start position of the first checkpoint log is denoted as backup_begin_lsn, and the start position of the second checkpoint log is denoted as backup_end_lsn. The position of the redo point (denoted as the redo point) recorded in the first checkpoint log can be denoted as begin_redo_lsn. The redo point recorded in the second checkpoint log is denoted as end_redo_lsn. The end position of the second checkpoint log is denoted as the end lsn.

[0094] On this basis, the range of the XLOG log generated during the base backup is denoted as: from begin_redo_lsn to the end lsn of the checkpoint log at backup_end_lsn. That is to say, the range information of the XLOG log generated by the database during the execution of the base backup operation is from begin_redo_lsn to the end lsn of the checkpoint log at backup_end_lsn.

[0095] The method for processing the transaction log of the database in this embodiment realizes the accurate acquisition of the range of the XLOG log generated during the base backup by obtaining the redo point recorded in the first checkpoint log created before the start of the base backup operation and the end position of the second checkpoint log created after the end of the base backup operation, and using the range between the redo point and the end position as the range information of the XLOG log generated by the database during the execution of the base backup operation, improving the accuracy of obtaining the page mirror data generated during the base backup.

[0096] In the above embodiment, the page mirror data corresponding to the XLOG log generated during the base backup still remains stored in the XLOG log and is not separated from the XLOG log.

[0097] In addition, when the base backup exists, the XLOG log carrying page data generated during the base backup is not allowed to be cleared.

[0098] In some embodiments, the above step S104 may include the following steps: obtaining the XLOG log generated by the database during the execution of the base backup operation according to the range information; obtaining the page mirror data retained in the XLOG log generated by the database during the execution of the base backup operation. Since the page mirror data corresponding to the XLOG log generated during the base backup is still stored in the XLOG log, the page mirror data can be directly obtained from the XLOG log generated by the database during the execution of the base backup operation.

[0099] Furthermore, the above step S106 may include the following steps: storing the XLOG log generated by the database during the execution of the base backup operation into the backup set, and the XLOG log stored in the backup set includes page mirror data.

[0100] The method for processing the transaction log of the database in this embodiment stores the XLOG log with page mirror data generated by the database during the execution of the base backup operation into the backup set, realizing the stable storage of the XLOG log with page mirror data generated during the base backup.

[0101] Figure 7 It is a schematic diagram of the structure of the page mirror file in the method for processing the transaction log of the database according to another embodiment of the present invention. As Figure 7 shown, in some embodiments, the page mirror data of the XLOG logs generated by the database before and after the execution of the base backup operation are respectively stored in two different page mirror files.

[0102] In this embodiment, after the above step S106, the method for processing the transaction log of the database of the present invention may further include the following steps: after the base backup operation ends, creating a page mirror file for storing the page mirror data in the XLOG log generated by the database after the base backup operation ends; separating the page mirror data and the XLOG log in the XLOG log generated by the database and storing them in the page mirror file.

[0103] For the page mirror file that was not completely written before the start of the base backup operation, the page mirror file can be directly switched, and after the base backup operation ends, the page mirror data in the XLOG log generated after the base backup operation ends is stored in the new page mirror file. The page mirror file that was not completely written before the start of the base backup operation can be cleaned up in the subsequent cleaning process, thus saving space.

[0104] In other embodiments, as Figure 5 shown, the page mirror data of the XLOG log generated by the database after the execution of the base backup operation can continue to be written into the page mirror file that was not completely written before the start of the base backup operation.

[0105] Figure 8 It is a schematic flowchart of a method for processing a transaction log of a database according to an embodiment of the present invention. The following will be combined with Figure 8 Specifically describe the flow steps of the method for processing the transaction log of the database in this embodiment.

[0106] Step S802: Obtain the range information of the XLOG log generated during the execution of the base backup operation of the database.

[0107] Step S804: Separate the page data in the XLOG log generated during the execution of the base backup operation of the database from the XLOG log to obtain the XLOG log after separating the page data.

[0108] Step S806: Store the page data separated from the XLOG log into the page data file;

[0109] Step S808: Obtain the page data generated during the execution of the base backup operation of the database according to the range information.

[0110] Step S810: Store the XLOG log after separating the page data generated during the execution of the base backup operation of the database and the corresponding page data into the backup set. This process ends.

[0111] Using the above method, by obtaining the range information of the XLOG log generated during the execution of the base backup operation of the database, according to the range information, obtaining the page mirror data generated during the execution of the base backup operation of the database, and storing the XLOG log and the page mirror data generated during the execution of the base backup operation of the database into the backup set respectively, the stable storage of the page mirror data during the base backup is realized. The method for processing the transaction log of the database of the present invention stores the page mirror data in the XLOG log generated during the base backup into the backup set, while ensuring the security of the base backup data, accurately retains the page mirror data required for page repair, and improves the stability of the database.

[0112] Figure 9 It is a schematic flowchart of a method for processing a transaction log of a database according to another embodiment of the present invention. The following will be combined with Figure 9 Specifically describe the flow steps of the method for processing the transaction log of the database in this embodiment.

[0113] Step S902: Obtain the range information of the XLOG log generated during the execution of the base backup operation of the database.

[0114] Step S904: Obtain the XLOG log carrying page data generated by the database during the execution of the base backup operation according to the range information.

[0115] Step S906: Store the XLOG log carrying page data generated by the database during the execution of the base backup operation into the backup set. This process ends.

[0116] Using the above method, by obtaining the range information of the XLOG log generated by the database during the execution of the base backup operation, according to the range information, obtaining the XLOG log carrying page mirror data generated by the database during the execution of the base backup operation, and storing the XLOG log carrying page mirror data generated by the database during the execution of the base backup operation into the backup set, the stable storage of the XLOG log and page mirror data during the base backup is achieved. The method for processing the transaction log of the database according to the present invention stores the XLOG log carrying page mirror data into the backup set, while ensuring the security of the base backup data, precisely retains the page mirror data required for page repair, and improves the stability of the database.

[0117] This embodiment also provides a machine-readable storage medium and a computer device. Figure 10 It is a schematic diagram of a machine-readable storage medium 80 according to an embodiment of the present invention. Figure 11 It is a schematic diagram of a computer device 90 according to an embodiment of the present invention.

[0118] The machine-readable storage medium 80 stores a machine-executable program 81 thereon. When the machine-executable program 81 is executed by a processor, it implements the method for processing the transaction log of the database in any of the above embodiments.

[0119] The computer device 90 may include a memory 920, a processor 910, and a machine-executable program 81 stored on the memory 920 and running on the processor 910. When the processor 910 executes the machine-executable program 81, it implements the method for processing the transaction log of the database in any of the above embodiments.

[0120] It should be noted that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any machine-readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices.

[0121] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For the description of this embodiment, the machine-readable storage medium 80 can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer device 90 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smart phone. The computer device 90 can include a processor 910 adapted to execute the stored instructions and a memory 920 that provides temporary storage space for the operation of the instructions during operation. The processor 910 can be a single-core processor, a multi-core processor, a computing cluster, or any other number of other configurations. The memory 920 can include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0122] It should be noted that in some alternative embodiments, the solution of the present invention is applicable to relational databases, and in particular to the KingbaseES database (abbreviated as KES database), which enriches the functions of the database and improves the efficiency of the database. In some other alternative embodiments, the method for processing the transaction log of the database of the present invention can also be applicable to other relational databases.

[0123] In addition, the flowcharts provided in this embodiment are not intended to indicate that the operations of the method will be executed in any specific order, or that all operations of the method are included in every case. In addition, the method can include additional operations. Within the scope of the technical concept provided by the method of this embodiment, additional changes can be made to the above method.

[0124] At this point, those skilled in the art should recognize that although multiple exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived from the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and determined to cover all these other variations or modifications.

Claims

1. A method for processing a transaction log of a database, comprising: Obtaining range information of the transaction log generated during a base backup operation of the database; Obtaining page mirror data generated during the base backup operation of the database according to the range information, where the page mirror data is data of a data page recorded in the transaction log; Storing the page mirror data generated during the base backup operation of the database into a backup set.

2. The method for processing a transaction log of a database according to claim 1, wherein, Before the step of obtaining page mirror data generated during the base backup operation of the database according to the range information, the method for processing the transaction log of the database further comprises: Separating the page mirror data in the transaction log generated during the base backup operation of the database from the transaction log to obtain a transaction log after separation of the page mirror data; Storing the page mirror data separated from the transaction log into a page mirror file, where the page mirror file is used to store page mirror data separated from the transaction log of the database.

3. The method for processing a transaction log of a database according to claim 2, wherein, The page mirror file includes a used interval and a to-be-written interval, and the used interval is used to store the page mirror data in the transaction log generated before the base backup operation of the database; And Storing the page mirror data separated from the transaction log into the page mirror file includes: Obtaining the to-be-written interval of the page mirror file; Storing the page mirror data in the transaction log generated during the base backup operation of the database into the to-be-written interval of the page mirror file.

4. The method for processing a transaction log of a database according to claim 2, wherein, After the step of obtaining page mirror data generated during the base backup operation of the database according to the range information, the method for processing the transaction log of the database further comprises: Storing the transaction log after separation of the page mirror data generated during the base backup operation of the database into the backup set.

5. The method for processing a transaction log of a database according to claim 1, wherein, Obtaining page mirror data generated during the base backup operation of the database according to the range information includes: Obtaining the transaction log generated during the base backup operation of the database according to the range information; Obtaining the page mirror data retained in the transaction log generated during the base backup operation of the database.

6. The method for processing a transaction log of a database according to claim 5, wherein, Storing the page mirror data generated during the base backup operation of the database into the backup set includes: Storing the transaction log generated during the base backup operation of the database into the backup set, and the transaction log stored in the backup set includes the page mirror data.

7. The method for processing a transaction log of a database according to claim 5, wherein, After the step of storing the page mirror data generated during the execution of the base backup operation in the database into the backup set, the method for processing the transaction log of the database further includes: After the base backup operation ends, create a page mirror file for storing the page mirror data in the transaction log generated by the database after the base backup operation ends; Separate the page mirror data in the transaction log generated by the database from the transaction log and store it in the page mirror file.

8. The method for processing the transaction log of the database according to claim 1, wherein Obtaining the range information of the transaction log generated by the database during the execution of the base backup operation includes: Obtaining the position of the redo point recorded in the first checkpoint log created before the start of the base backup operation; Obtaining the end position of the second checkpoint log created after the base backup operation ends; Taking the range between the position of the redo point and the end position as the range information of the transaction log generated by the database during the execution of the base backup operation.

9. A machine-readable storage medium having a machine-executable program stored thereon, the machine-executable program, when executed by a processor, implements the method for processing the transaction log of the database according to any one of claims 1 to 8.

10. A computer device, including a memory, a processor, and a machine-executable program stored on the memory and running on the processor, and when the processor executes the machine-executable program, it implements the method for processing the transaction log of the database according to any one of claims 1 to 8.

Images