Log data backup method and device, equipment, storage medium and program product

Abstract

The invention relates to a log data backup method and device, equipment, a storage medium and a program product, and relates to the technical field of databases. The method comprises the steps of writing to-be-archived log data into archived log fragments in a cache region, and stopping data writing of the current archived log fragment and writing the remaining log data into the next archived log fragment under the condition that a target fragment switching condition is met until data writing of the to-be-archived log data is completed; in response to a query command for log backup, determining a target log backup range according to the query command and a system change number of a database; according to the target log backup range, determining a target archiving log fragment in each archiving log fragment; and copying and storing the target archiving log fragment into a backup area. By adopting the method, the problem of archiving log loss caused by sharing the storage path can be avoided, and the completeness of the backup chain is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, and in particular, to a log data backup method and device, a computer device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] For an OceanBase database, a traditional log management mode is to use a distributed architecture. Under the distributed architecture, an archive log is generated by a log stream leader node, and a storage path of the archive log is specified by a LOG_ARCHIVE_DEST parameter. The distributed architecture does not have a separate archive log backup storage, but directly sets the archive log storage path as the backup storage to realize backup.

[0003] However, the archive log and the backup storage share the same storage path, and the OceanBase database is prone to losing the archive log due to business operation and maintenance deletion or automatic cleaning operation, thereby causing a backup chain to be broken, triggering an incremental backup to fail, being unable to obtain a complete log sequence, and subsequent incremental backup being unable to normally proceed, resulting in incomplete backup data and being able to only be restored to a latest complete backup point, being unable to support point-in-time recovery (PITR), and being unable to meet the demand when precise recovery to a specific time point is required. SUMMARY

[0004] Therefore, it is necessary to provide a log data backup method, device, computer device, computer readable storage medium and computer program product in view of the above technical problems.

[0005] In a first aspect, the present application provides a log data backup method, comprising:

[0006] writing log data to be archived into an archive log segment in a cache area, stopping data writing to a current archive log segment and writing remaining log data to a next archive log segment in a case where a target segment switching condition is met, and until data writing to the log data to be archived is completed;

[0007] in response to a query command for log backup, determining a target log backup range according to the query command and a system change number of a database;

[0008] determining a target archive log segment in each of the archive log segments according to the target log backup range;

[0009] copying and storing the target archive log segment into a backup area.

[0010] In one of the embodiments, the target log backup range is determined according to the query command and the system change number of the database in response to the query command for the log backup, comprising:

[0011] In response to the query command for the log backup, the checkpoint system change number and the starting system change number corresponding to the target archive log segment of the last successful backup are obtained;

[0012] According to the query command, the checkpoint system change number and the starting system change number, the identification information and the physical path of one or more archive log segments that need to be backed up are determined;

[0013] According to the identification information and the physical path of the one or more archive log segments that need to be backed up, the starting system change number of the target log backup range is determined.

[0014] In one of the embodiments, the log data to be archived is written into the archive log segment in the cache area, and the data writing of the current archive log segment is stopped and the remaining log data is written into the next archive log segment under the condition that the target segment switching condition is met, until the data writing of the log data to be archived is completed, comprising:

[0015] The log data to be archived is written into the redo log for each archive log segment in the cache area;

[0016] Under the condition that the target log switching condition is met, the data writing of the current redo log is stopped, the current redo log is archived to the current archive log segment, and the remaining log data is written into the next redo log, until the target segment switching condition is met, then the data writing of the current archive log segment is stopped and the remaining log data is written into the next archive log segment.

[0017] In one of the embodiments, the data writing of the current redo log is stopped under the condition that the target log switching condition is met, comprising:

[0018] In the case that the data writing time of the current redo log reaches the target time threshold, the data writing of the current redo log is stopped;

[0019] Or,

[0020] In the case that the total log capacity of the current redo log reaches the target size threshold, the data writing of the current redo log is stopped.

[0021] In one of the embodiments, the target segment switching condition comprises that the data writing time of the current archive log segment reaches the target rotation period, or the total log capacity of the current archive log segment reaches the target value.

[0022] In one of the embodiments, the method further comprises:

[0023] copying and storing each of the target archive log segments stored in the backup area into the cache area in response to a log data recovery request for the cache area.

[0024] In a second aspect, the present application provides a log data backup device, comprising:

[0025] a writing module, configured to write log data to be archived into an archive log segment in a cache area, stop data writing to a current archive log segment and write remaining log data into a next archive log segment in a case where a target segment switching condition is met, and complete data writing to the log data to be archived;

[0026] a first response module, configured to determine a target log backup range according to a query command for log backup and a system change number of a database in response to the query command;

[0027] a determination module, configured to determine a target archive log segment from the archive log segments according to the target log backup range;

[0028] a backup module, configured to copy and store the target archive log segment into a backup area.

[0029] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0030] write log data to be archived into an archive log segment in a cache area, stop data writing to a current archive log segment and write remaining log data into a next archive log segment in a case where a target segment switching condition is met, and complete data writing to the log data to be archived;

[0031] determine a target log backup range according to a query command for log backup and a system change number of a database in response to the query command;

[0032] determine a target archive log segment from the archive log segments according to the target log backup range;

[0033] copy and store the target archive log segment into a backup area.

[0034] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0035] writing log data to be archived into an archive log segment in a cache area, stopping data writing to a current archive log segment and writing remaining log data to a next archive log segment if a target segment switching condition is met, until data writing to the log data to be archived is completed;

[0036] in response to a query command for a log backup, determining a target log backup range according to the query command and a system change number of a database;

[0037] determining a target archive log segment from the archive log segments according to the target log backup range;

[0038] copying and storing the target archive log segment into a backup area.

[0039] In a fifth aspect, the present application provides a computer program product, comprising a computer program which, when executed by a processor, implements the following steps:

[0040] writing log data to be archived into an archive log segment in a cache area, stopping data writing to a current archive log segment and writing remaining log data to a next archive log segment if a target segment switching condition is met, until data writing to the log data to be archived is completed;

[0041] in response to a query command for a log backup, determining a target log backup range according to the query command and a system change number of a database;

[0042] determining a target archive log segment from the archive log segments according to the target log backup range;

[0043] copying and storing the target archive log segment into a backup area.

[0044] The aforementioned log data backup method, apparatus, computer equipment, computer-readable storage medium, and computer program product write the log data to be archived into archived log segments in a buffer area. When the target segment switching condition is met, data writing to the current archived log segment is stopped, and the remaining log data is written to the next archived log segment until the data writing of the log data to be archived is completed. In response to a query command for log backup, the target log backup range is determined based on the query command and the database system change number. Based on the target log backup range, the target archived log segments in each archived log segment are determined. The target archived log segments are copied and stored in the backup area. Compared to traditional methods, this solution separates archived log data from backup storage by writing the log data to be archived into archived log fragments in the cache. When certain conditions are met, fragment switching occurs, and the target archived log fragment is copied and stored in the backup area. This avoids archived log loss due to shared storage paths, and the archived logs in the backup area are less prone to loss. Even if the necessary log data in the cache is cleared, data recovery can be achieved using the log data in the backup area, ensuring the integrity of the backup chain. This prevents incremental backup failures caused by archived log loss, ensuring the complete log sequence can be obtained, allowing subsequent incremental backups to proceed normally and guaranteeing the integrity of the backup data. Furthermore, the target log backup range corresponding to the target archived log fragment in the backup area is associated with the database's system change number, and the backup chain is complete. This allows for point-in-time recovery of log data based on the system change number, improving the accuracy of log data recovery. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is an application environment diagram of a log data backup method in one embodiment;

[0047] Figure 2 This is a flowchart illustrating a log data backup method in one embodiment;

[0048] Figure 3 This is a schematic diagram of a log hierarchical partitioned isolated storage architecture in one embodiment;

[0049] Figure 4 This is a flowchart illustrating a log data backup method in another embodiment;

[0050] Figure 5 Figure 1 is a structural block diagram of a log data backup device according to an embodiment of the present application.

[0051] Figure 6 Figure 2 is an internal structural diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0053] It should be noted that the terms "first", "second", and the like used in the present application can be used to describe various objects, but these objects are not limited by these terms. These terms are only used to distinguish the first object from the second object. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "multiple" used in the present application refers to two or more. The term "and / or" used in the present application refers to one of the options or any combination of multiple options.

[0054] Embodiments of the present application relate to the following terms:

[0055] SCN (System Change Number): refers to a globally unique incremental sequence number used to identify system changes in a database, which can be used to reflect the time order of transactions and log changes in the database. Each transaction submission or log writing corresponds to a unique SCN, and the value of SCN is continuously incremented as the system runs, so it can be used to represent the state of the database at a certain time.

[0056] Log Stream: is a logical unit for managing continuously increasing Redo (redo) logs in OceanBase database, which can be used to record all change operations (such as transaction submission, data modification, etc.) of the database. Each log stream implements multi-copy synchronization through Paxos (Paxos) protocol to ensure high availability and consistency of log data.

[0057] Leader: Leader is a special copy in the log stream, responsible for processing all write requests, and synchronizing logs to other copies (Follower) through Paxos protocol. Each log stream has only one Leader at any time.

[0058] Piece: Piece is the physical organization unit of the archive log, which is used to represent the complete log set generated by a certain tenant in a continuous time period, including the archive log piece. Its time range is controlled by the parameter piece_switch_interval (piece switch interval) (such as 1 day), which ensures the continuity and completeness of the log.

[0059] SYNC_SCN (Synchronized System Change Number): It is used to represent the latest SCN value of all replicas in the log stream that have completed synchronization. When the leader node writes the log, it will wait for at least half of the replicas to confirm synchronization before updating this value, ensuring data strong consistency.

[0060] START_SCN (Start System Change Number): It refers to the starting system change number of the log piece, which is used to identify the SCN of the first valid log entry of the archive log piece. The START_SCN of each piece is globally unique and increasing, which is used to define the cold and hot data boundary.

[0061] END_SCN (End System Change Number): It refers to the ending system change number of the log piece, which is used to identify the SCN of the last valid log entry of the archive log piece.

[0062] The log data backup method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 . Among them, the terminal communicates with the server through the network. The terminal can be, but is not limited to, various personal computers, notebook computers, tablet computers, unmanned aerial vehicles, low-altitude flying vehicles, and Internet of Things devices. The Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, a smart vehicle device, a projection device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0063] In an exemplary embodiment, as shown in Figure 2 , a log data backup method is provided, which can be applied to a server in Figure 1 . The method can include the following steps:

[0064] Step 201, write the log data to be archived into an archive log segment in the cache area, stop writing data to the current archive log segment and write the remaining log data into the next archive log segment if the target segment switching condition is met, until the data writing of the log data to be archived is completed.

[0065] The log data to be archived refers to the log data that needs to be written into the cache area. The cache area is used to temporarily store the log data to be archived or backed up, provides an independent space for the preliminary storage of the log, ensures that the log can be safely stored after being generated, is not affected by the backup software, is separated from the backup storage path, and avoids business cleaning interference. The target segment switching condition refers to the condition that needs to switch the archive log segment. The remaining log data refers to the log data to be archived that is not written into the cache area.

[0066] Exemplarily, the log data to be archived can be written into the cache area by the database of the OceanBase cluster (i.e., the OceanBase database). The OceanBase database adopts a distributed log architecture. Specifically, according to the LOG_ARCHIVE_DEST parameter in the configuration parameter of the OceanBase cluster, the cache area directory in the backup storage system is pointed to and queried, and the cache area storage path corresponding to the log data to be archived is determined according to the query result. According to the cache area storage path, an archive log segment is created in the cache area, the log data to be archived is written into the archive log segment, the data writing of the archive log segment is stopped, a new archive log segment is created in the cache area, and the remaining log data is written into the new archive log segment if the target segment switching condition is met, until the data writing of the log data to be archived is completed.

[0067] In this embodiment, the log stream Leader node generates an archive log, determines the log data to be archived according to the archive log, and immediately writes the log data to be archived into the cache area, which is not perceived by the business.

[0068] Step 202, in response to a query command for log backup, determine the target log backup range according to the query command and the system change number of the database.

[0069] The query command is used to query the name of the archive log segment falling within the target log backup range. The target log backup range refers to the range in which one or more archive log segments to be backed up are located.

[0070] Exemplarily, when a preset log backup time point (such as 2 a.m. every day) is reached, a query command for log backup is triggered, the name of an archived log segment falling within a target log backup range is queried according to a query instruction and a system change number of the database, and the target log backup range is determined. Precise positioning in the OceanBase database is implemented by the following SCN characteristics of the OceanBase cluster: (1) monotonicity: all SCNs are strictly increasing in time line, ensuring that log segments are arranged in chronological order; (2) global consistency: the final consistency of SYNC_SCN among multiple copies is ensured through the Paxos protocol; and (3) boundary identification: the START_SCN and the END_SCN of each piece form a closed interval, forming an immutable log unit. Based on the above SCN characteristics, the Last_Backup_SCN (Last_Backup_System Change Number, last successful end system change number of backup) of a target archived log segment of the last successful backup, the current SYNC_SCN and the Current_SCN (the latest SCN of the current system) are obtained, the maximum value of the Last_Backup_SCN and the current SYNC_SCN is taken as the starting end of the target log backup range, and the Current_SCN is taken as the ending end of the target log backup range, so as to determine the target log backup range. According to the target log backup range, one or more archived log segments that need to be backed up, i.e., target archived log segments, can be precisely positioned.

[0071] In the embodiment, the archived concurrency can be adjusted through the log_archive_concurrency parameter in the configuration parameter of the OceanBase cluster, and concurrent performance of multiple log archiving backups can be achieved according to the archived concurrency, so as to avoid resource competition.

[0072] In step 203, the target archived log segment in each archived log segment is determined according to the target log backup range.

[0073] The target archived log segment refers to one or more archived log segments that need to be backed up. The target archived log segment in each archived log segment can refer to a segment composed of all archived log segments in each archived log segment that meet the target log backup range.

[0074] For example, the target log backup range is the range from 1653999999000100000 to 16540000000000000000 in the archived log segment named piece_d1001r2 and the archived log segment named piece_d1001r3; based on the target log backup range, the target archived log segments are determined to be the incremental portion from 1653999999000100000 to 16540000000000000000 in piece_d1001r2 and the full log of piece_d1001r3.

[0075] Step 204: Copy the target archived log fragment and save it to the backup area.

[0076] The backup area is used to store backup log data.

[0077] For example, the target archived log segment is the incremental portion from 1653999999000100000 to 16540000000000000000 in piece_d1001r2, and the full log of piece_d1001r3. The incremental portion from 1653999999000100000 to 16540000000000000000 in piece_d1001r2, along with the full log of piece_d1001r3, is copied and stored in the storage path within the backup area. Specifically, only the incremental log after 1653999999000100000 in piece_d1001r2 is copied, avoiding duplicate backups of already fully backed-up historical logs and improving backup efficiency.

[0078] For example, such as Figure 3 As shown, the target archived log fragments are the incremental part of piece_d1001r1 and the full log of piece_d1001r2. The incremental part of piece_d1001r1 and the full log of piece_d1001r2 are copied and stored in the storage path in the backup area. The name of this storage path is backup_set_3_log.

[0079] In this embodiment, after copying the target archived log fragment and storing it in the backup area, a checksum (such as SHA-256) is generated and compared with the metadata of the OceanBase database (the aforementioned log data to be archived) to ensure that the log data in the backup area is intact. Furthermore, expired logs can be automatically deleted according to backup area retention rules (such as retaining logs from the most recent 7 days) to avoid storage bloat.

[0080] In the log data backup method, the log data to be archived is written into an archive log segment in a cache area, data writing to the current archive log segment is stopped and remaining log data is written into a next archive log segment when a target segment switching condition is met, and data writing to the log data to be archived is completed; in response to a query command for log backup, a target log backup range is determined according to the query command and a system change number of the database; target archive log segments in each archive log segment are determined according to the target log backup range; and the target archive log segments are copied and stored in a backup area. Compared with the traditional method, the present application writes the log data to be archived into the archive log segment in the cache area, switches the segments when the condition is met, copies the target archive log segments and stores them in the backup area, realizes the separation of the archive log and the backup storage, avoids the problem of loss of the archive log due to the shared storage path, and the archive log in the backup area is not easy to lose. Even if the log data to be retained in the cache area is cleaned up, data recovery can be realized by using the log data in the backup area, the integrity of the backup chain can be ensured, the problem of invalid incremental backup caused by loss of the archive log can be avoided, complete log sequences can be ensured, and subsequent incremental backup can be normally performed, thereby ensuring the integrity of the backup data. In addition, the target log backup range corresponding to the target archive log segment in the backup area is associated with the system change number of the database, and the backup chain is complete, time point recovery of the log data can be realized according to the system change number, and the recovery accuracy of the log data is improved.

[0081] In one exemplary embodiment, in step 202, in response to a query command for log backup, a target log backup range is determined according to the query command and a system change number of the database, which can include:

[0082] In response to the query command for log backup, a checkpoint system change number and a starting system change number corresponding to a target archive log segment of the last successful backup are obtained; identification information and a physical path of one or more archive log segments that need to be backed up are determined according to the query command, the checkpoint system change number and the starting system change number; and a target log backup range is determined according to the identification information and the physical path of the one or more archive log segments that need to be backed up.

[0083] The target archive log segment of the last successful backup refers to the target archive log segment that is successfully backed up to the backup area last time. The checkpoint system change number corresponding to the target archive log segment is used to indicate an end point of backup of the target archive log segment. The starting system change number corresponding to the target archive log segment is used to indicate a starting point of backup of the target archive log segment. The identification information and the physical path of the target archive log segment are used to indicate the name of the target archive log segment.

[0084] Exemplarily, when a preset log backup time point (e.g., 2 a.m. every day) is reached, a query command for log backup is triggered, and a process of acquiring a target log backup range in response to the query command can include the following steps:

[0085] In response to the query command for log backup, the backup system queries, through a CDB_OB_LS (Cluster Database OceanBase Log Stream) view of the OceanBase cluster, a latest SCN of the current database as 1655000001000000000.

[0086] Through a CDB_OB_BACKUP_SET_FILES (Cluster Database OceanBase Backup Set Files) view, a CHECKPOINT_SCN (Check Point System Change Number) corresponding to the target archive log piece piece_d1001r2 of the last successful backup is queried as 1653999999000100000 and a START_SCN is 1653999999000000000.

[0087] Through the query command, identification information and a physical path (PATH) corresponding to one or more archive log pieces that need to be backed up are queried; according to the identification information and the physical path corresponding to the one or more archive log pieces that need to be backed up and the CHECKPOINT_SCN, a target log backup range in a cache area is determined as a range of 1653999999000100000 to 1654000000000000000 in piece_d1001r2 and piece_d1001r3; the identification information includes an identifier (TENANT_ID) of a database tenant, an identifier (DEST_ID) of a storage location, an identifier (ROUND_ID) of a round, and an identifier (PIECE_ID) of an archive log piece; the identification information and the physical path corresponding to the archive log piece are used to indicate a name and a location of the archive log piece.

[0088] The query command can be a SQL statement as follows:

[0089] SELECT TENANT_ID, DEST_ID, ROUND_ID, PIECE_ID, START_SCN, END_SCN,PATH

[0090] FROM CDB_OB_ARCHIVELOG_PIECE_FILES

[0091] WHERE START_SCN <= 1653999999000000000 -- The SCN from the last backup

[0092] AND END_SCN > 1653999999000100000;

[0093] In an exemplary embodiment, step 201, writing the log data to be archived into the archived log segment in the buffer, stopping the writing of data to the current archived log segment and writing the remaining log data to the next archived log segment when the target segment switching condition is met, until the writing of the log data to be archived is completed, may include:

[0094] For each archived log segment in the buffer, the log data to be archived is written to the redo log. If the target log switching condition is met, the writing of data to the current redo log is stopped, the current redo log is archived to the current archived log segment, and the remaining log data is written to the next redo log segment. This process continues until the target segment switching condition is met, at which point the writing of data to the current archived log segment is stopped, and the remaining log data is written to the next archived log segment.

[0095] The target log switching condition refers to the conditions under which the redo log needs to be switched. The target log switching condition can be that the total size of the current redo log reaches the target size threshold (i.e., the current redo log is full), or the data writing time of the current redo log reaches the target time threshold.

[0096] For example, in response to user input or operation, the configuration parameters of the OceanBase cluster are modified so that the LOG_ARCHIVE_DEST parameter in the OceanBase cluster configuration parameters points to the cache storage path; in response to the log stream leader node generating archived logs (at which point the OceanBase cluster is in log writing state), the log data to be archived is determined based on these archived logs. For this purpose, as follows... Figure 3 As shown, the process of sequentially writing log data to be archived into multiple archived log fragments in the buffer may include the following steps:

[0097] Based on the cache storage path, create an archived log fragment in the cache. The name of this archived log fragment is piece_d1001r2, and the START_SCN (Starting System Change Number) of piece_d1001r2 is 1653999999000000000.

[0098] The log data to be archived is written into a current redo log in the OceanBase database, and the name of the redo log is clog2.

[0099] When the total capacity of the log of clog2 reaches a target size threshold (such as 1 GB) or the data writing time of clog2 reaches a target time threshold (such as 30 minutes), a redo log switching mechanism is triggered, under the action of which the data writing to clog2 is stopped, the log stream Leader node archives clog2 to piece_d1001r2 (that is, writes the log data carried by clog2 into piece_d1001r2), a new redo log (the name of the redo log is clog3) is created in the OceanBase database, and the remaining log data is written into clog3.

[0100] When the total capacity of the log of clog3 reaches a target size threshold (such as 1 GB) or the data writing time of clog3 reaches a target time threshold (such as 30 minutes), the redo log switching mechanism is triggered again, under the action of which the data writing to clog3 is stopped, the log stream Leader node archives clog3 to piece_d1001r2, a new redo log (the name of the redo log is clog4) is created in the OceanBase database, and the remaining log data is written into clog4.

[0101] During the data writing process of clog4, if the target segment switching condition is met, clog4 is archived to piece_d1001r2, the data writing to piece_d1001r2 is stopped (that is, piece_d1001r2 no longer receives the writing of a new redo log), a new archived log segment (the name of the archived log segment is piece_d1001r3) is created in the cache area, and the remaining log data continues to be written into clog4, and the START_SCN of piece_d1001r3 is 1654000000000000000.

[0102] When the total capacity of the log of clog4 reaches a target size threshold (such as 1 GB) or the data writing time of clog4 reaches a target time threshold (such as 30 minutes), the redo log switching mechanism is triggered again, under the action of which the data writing to clog4 is stopped, the log stream Leader node archives clog4 to piece_d1001r3, a new redo log (the name of the redo log is clog5) is created in the OceanBase database, and the remaining log data is written into clog5.

[0103] In the data writing process of the clog5, if the to-be-archived log data is completely written, the clog5 is archived to the piece_d1001r3 and the data writing to the piece_d1001r3 is stopped (that is, the piece_d1001r3 no longer receives the writing of new redo logs), at this time, the data writing of the to-be-archived log data is completed.

[0104] The archived log segment can be marked as hot data after being created, and the archived log segment can be marked as cold data after the data writing to the archived log segment is stopped. The hot data can be used to accept data writing, and the cold data is only used to meet the historical backup or recovery demand. The macro block of the OceanBase database can be used to improve the storage efficiency of the OceanBase database.

[0105] In an example embodiment, in the case of meeting the target log switching condition, stopping the data writing to the current redo log can include:

[0106] In the case that the data writing time of the current redo log reaches a target time threshold, the data writing to the current redo log is stopped; or in the case that the total log capacity of the current redo log reaches a target size threshold, the data writing to the current redo log is stopped.

[0107] The target log switching condition can include that the data writing time of the current redo log reaches a target time threshold, or the total log capacity of the current redo log reaches a target size threshold. The target size threshold and the target time threshold can be respectively corresponding to one target size threshold and one target time threshold for each redo log, or the target size threshold and the target time threshold can be commonly corresponding to one target size threshold and one target time threshold for all redo logs.

[0108] For example, the target time threshold is 30 minutes and the target size threshold is 1 GB. In the case that the data writing time of the current redo log reaches 30 minutes, the data writing to the current redo log is stopped; in the case that the total log capacity of the current redo log reaches 1 GB (indicating that the current redo log is full), the data writing to the current redo log is stopped.

[0109] In an example embodiment, the target segment switching condition can include that the data writing time of the current archived log segment reaches a target rotation period, or the total log capacity ratio of the current archived log segment reaches a target value.

[0110] Exemplarily, the target rotation period is 2 hours, and the target value is 80%. When the data writing time of the current archive log segment reaches 2 hours, the data writing to the current archive log segment is stopped, and the remaining log data is written to the next archive log segment (that is, the data writing object is switched from the current archive log segment to the next archive log segment); when the log total capacity ratio of the current archive log segment reaches 80%, the data writing to the current archive log segment is stopped, and the remaining log data is written to the next archive log segment.

[0111] In an exemplary embodiment, the method of the embodiments of the present application can further include a log data recovery step, which can include:

[0112] In response to a log data recovery request for the cache area, each target archive log segment stored in the backup area is copied and stored in the cache area.

[0113] Exemplarily, when the log cleaning task trigger condition is met, the system automatically performs log data cleaning. The log cleaning task trigger condition can be that the cache area space usage rate exceeds 90% or exceeds the log retention period of 7 days. If it is detected that the log data of the cache area is abnormally cleaned (data that should not be cleaned is cleaned, which belongs to abnormal cleaning), a log data recovery request for the cache area can be automatically initiated, or a log data recovery request for the cache area is generated according to user input, and in response to the log data recovery request for the cache area, each target archive log segment stored in the backup area is copied and stored in the cache area.

[0114] Currently, OceanBase database uses a distributed log architecture, and the archive log is generated by the log stream Leader node, and the storage path is specified through the LOG_ARCHIVE_DEST parameter. There is no separate archive log backup in the original design, and backup is achieved by setting the archive log storage path as a backup storage. However, the key limitation is that the archive log and the backup storage share the storage path, and there is a risk of backup chain rupture caused by maintenance misdeletion or automatic cleaning. The traditional backup scheme (such as full backup + incremental backup) depends on the complete log chain, and has the following technical problems: ① Storage coupling risk: the archive log and the backup storage path are not separated, and business operation (such as manual cleaning) may misdelete backup data; ② Insufficient recovery granularity: dependent on the continuity of log archiving, if the intermediate log is missing, accurate recovery cannot be achieved by combining backup sets. Once the archive log is cleaned by the business, the following consequences will be caused: (1) Incremental backup failure: the complete log sequence cannot be obtained, and subsequent incremental backup cannot be normally performed, resulting in incomplete backup data; (2) Recovery capability degradation: only the latest complete backup point can be restored, and PITR cannot be supported, which cannot meet the demand for accurate recovery to a specific time point.

[0115] To this end, in one exemplary embodiment, as Figure 4 shown, a log data backup method is also provided, which can be applied to a server in Figure 1 The method can include the following steps:

[0116] Step 401, modify the LOG_ARCHIVE_DEST parameter in the configuration parameter of the OceanBase cluster; the LOG_ARCHIVE_DEST parameter points to the directory of the cache area in the backup storage system; the directory of the cache area is used to indicate the cache storage path.

[0117] The backup storage system includes a cache area and a backup area that are in communication connection; the business log stored in the OceanBase database and the cache area is isolated from the backup data stored in the backup area. The cache area uses high-performance storage, and the backup area uses distributed storage.

[0118] Step 402, after the log stream leader node generates an archive log, the log data to be archived is determined according to the archive log, the log data to be archived is written into the redo log for each archive log segment in the cache area, the data writing to the current redo log is stopped under the condition that the target log switching condition is met, the current redo log is archived to the current archive log segment, and the remaining log data is written into the next redo log, until the target segment switching condition is met, then the data writing to the current archive log segment is stopped and the remaining log data is written into the next archive log segment.

[0119] Step 403, in response to a query command for log backup, the checkpoint system change number and the starting system change number corresponding to the target archive log segment of the last successful backup are obtained.

[0120] Step 404, according to the query command, the checkpoint system change number and the starting system change number, the identification information and the physical path of one or more archive log segments that need to be backed up are determined.

[0121] Step 405, according to the identification information and the physical path of one or more archive log segments that need to be backed up, the target log backup range is determined.

[0122] Step 406, according to the target log backup range, the target archive log segment in each archive log segment is determined.

[0123] Exemplarily, the target archive log segment can be the full log of one or more archive log segments within the target log backup range, or can be the partial incremental log within the target log backup range in the archive log segment.

[0124] Step 407, copying and storing the target archived log segment into the backup area.

[0125] Step 408, copying and storing each target archived log segment stored in the backup area into the cache area in response to the log data recovery request for the cache area. Illustratively, when the cache area space usage rate exceeds 90% or exceeds the log retention period of 7 days, the server cleans up part of the log data in the database and / or the cache area. During the cleaning process, some log data that still needs to be retained may be mistakenly cleaned up, so by initiating the log data recovery request for the cache area to the server, the server copies and stores each target archived log segment stored in the backup area into the cache area in response to the log data recovery request for the cache area.

[0126] In this embodiment, by isolating the archived log data in the cache area from the backup data backed up to the backup area, even if the log data that needs to be retained in the cache area is cleaned up, log data recovery can be performed through the complete backup data of the backup area, the risk of log data deletion can be avoided, data consistency and recoverability can be ensured, and by copying and storing each target archived log segment stored in the backup area into the cache area in response to the log data recovery request for the cache area, any time point recovery can be supported, thereby refining the recovery granularity of log data to the second level. In addition, the cache area of the present scheme uses high-performance storage, which can reduce the competition for database disk IO, and the backup area of the present scheme uses distributed storage, which can improve the throughput and support TB-level log real-time processing.

[0127] It should be understood that although each step in the flowchart involved in each of the above-described embodiments is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above-described embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0128] Based on the same inventive concept, the embodiments of the present application also provide a log data backup device for implementing the log data backup method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more log data backup device embodiments provided below can refer to the limitations of the log data backup method described above, which will not be described here.

[0129] In one exemplary embodiment, as shown in Figure 5 , a log data backup device is provided, which can be applied to a server in Figure 1 , and the device can include:

[0130] The writing module 501 is configured to write the log data to be archived into an archive log segment in the cache area, stop writing data into the current archive log segment and write the remaining log data into the next archive log segment when the target segment switching condition is met, and complete the data writing of the log data to be archived.

[0131] The first response module 502 is configured to determine a target log backup range according to the query command and the system change number of the database in response to the query command for log backup.

[0132] The determination module 503 is configured to determine a target archive log segment in each archive log segment according to the target log backup range.

[0133] The backup module 504 is configured to copy and store the target archive log segment into the backup area.

[0134] In one exemplary embodiment, the first response module 502 is further configured to obtain a checkpoint system change number and a starting system change number corresponding to the target archive log segment of the last successful backup in response to the query command for log backup, determine the identification information and the physical path of one or more archive log segments that need to be backed up according to the query command, the checkpoint system change number and the starting system change number, and determine the target log backup range starting system change number according to the identification information and the physical path of the one or more archive log segments that need to be backed up.

[0135] In one exemplary embodiment, the writing module 501 is further configured to write the log data to be archived into a redo log for each archive log segment in the cache area, stop writing data into the current redo log, archive the current redo log to the current archive log segment, and write the remaining log data into the next redo log when the target log switching condition is met, and stop writing data into the current archive log segment and write the remaining log data into the next archive log segment when the target segment switching condition is met.

[0136] In an example embodiment, the writing module 501 is further configured to stop writing data to the current redo log when a data writing time of the current redo log reaches a target time threshold, or stop writing data to the current redo log when a total capacity of the current redo log reaches a target size threshold.

[0137] In an example embodiment, the target segment switching condition includes that a data writing time of the current archive log segment reaches a target rotation period, or a total capacity of the current archive log segment reaches a target value.

[0138] In an example embodiment, the log data backup apparatus can further include a second response module. The second response module is configured to, in response to a log data recovery request for the cache area, copy and store each target archive log segment stored in the backup area into the cache area.

[0139] Each module in the log data backup apparatus described above can be implemented in whole or in part by software, hardware, or a combination thereof. Each module described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform operations corresponding to each module.

[0140] In an example embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 6 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store OceanBase cluster configuration parameters, redo logs, and other data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a log data backup method.

[0141] Those skilled in the art can understand that Figure 6The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0142] In an exemplary embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the log data backup method described above when executing the computer program.

[0143] In an embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the log data backup method described above when executed by a processor.

[0144] In an embodiment, a computer program product is provided, comprising a computer program, and the computer program implementing the log data backup method described above when executed by a processor.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0146] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0147] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A log data backup method characterized by comprising: The method comprises: writing log data to be archived into an archive log segment in a cache area, stopping data writing to a current archive log segment and writing remaining log data into a next archive log segment if a target segment switching condition is met, until data writing to the log data to be archived is completed; in response to a query command for a log backup, determining a target log backup range according to the query command and a system change number of a database; determining a target archive log segment in each of the archive log segments according to the target log backup range; copying and storing the target archive log segment into a backup area.

2. The method of claim 1, wherein, The method further comprises: in response to a query command for a log backup, obtaining a checkpoint system change number and a starting system change number corresponding to a target archive log segment of a last successful backup; determining identification information and a physical path of one or more archive log segments that need to be backed up according to the query command, the checkpoint system change number and the starting system change number; determining a target log backup range according to the identification information and the physical path of the one or more archive log segments that need to be backed up.

3. The method of claim 1, wherein, The method further comprises: for each of the archive log segments in the cache area, writing log data to be archived into a redo log; stopping data writing to a current redo log, archiving the current redo log to a current archive log segment, and writing remaining log data into a next redo log if a target log switching condition is met, until data writing to the current archive log segment is stopped and remaining log data is written into a next archive log segment if the target segment switching condition is met.

4. The method of claim 3, wherein, The method further comprises: stopping data writing to the current redo log if a data writing time of the current redo log reaches a target time threshold value; or stopping data writing to the current redo log if a total log capacity of the current redo log reaches a target size threshold value.

5. The method of claim 1, wherein, The target segment switching condition comprises that a data writing time of the current archive log segment reaches a target rotation period, or a total log capacity of the current archive log segment reaches a target value.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: in response to a log data recovery request for the cache area, copying and storing each of the target archive log segments stored in the backup area into the cache area.

7. A log data backup apparatus characterized by comprising: The apparatus comprises: The writing module is configured to write log data to be archived into an archived log segment in a cache area, stop writing data to a current archived log segment and write remaining log data to a next archived log segment until the writing of the log data to be archived is completed, when a target segment switching condition is met. The first response module is configured to, in response to a query command for a log backup, determine a target log backup range according to the query command and a system change number of a database. The determination module is configured to determine a target archived log segment from the archived log segments according to the target log backup range. The backup module is configured to copy and store the target archived log segment into a backup area.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

Images