Method, system, device and storage medium for database continuous log backup

Abstract

The application discloses a kind of database continuous log backup method, system, device and storage medium, belong to log backup technical field, comprising: query archive log list, copy all archive log files;Copy all redo log files, generate redo log copy file, obtain the path of the redo log file being written currently and the offset of the file, read the current redo log content from the start position to the offset position, overwrite the corresponding copy file, to calibrate the current redo log copy file content;If offset position changes, then read the content between last offset and current offset from current redo log file, overwrite to the corresponding copy file, the database continuous log backup method, system, device and storage medium do not need to install drive, have no performance influence to system and application, will not cause system crash downtime, without additional preparation target machine in advance, reduce implementation cost.

Description

Technical Field

[0001] This invention belongs to the field of log backup technology, specifically relating to methods, systems, devices, and storage media for continuous log backup of databases. Background Technology

[0002] There are currently three methods for backing up Oracle databases:

[0003] The first method currently available utilizes Oracle's built-in backup interface. However, this interface only backs up data files and archive logs, not redo logs. As a result, some redo log data will be lost during recovery, which is unacceptable for the banking and other financial industries.

[0004] The second method, Continuous Data Protection (CDP), backs up data at the disk sector level and installs drivers to capture and back up system and application data changes in real time to achieve the purpose of backing up Oracle. This method requires installing drivers on the production system. The driver intercepts the changed data in the system, makes a copy and saves it, and then allows the changed data to pass through. The slower the data is allowed through, the greater the impact on the performance of Oracle applications. In addition, if the driver has problems and is unstable, it may cause the production system to crash and shut down, resulting in business interruption.

[0005] The existing third method, Oracle synchronization technology, requires an additional target machine with Oracle installed. It parses the source Oracle logs to obtain SQL statements and then replays the Oracle operation records sequentially on the target machine to achieve Oracle backup. However, this solution only guarantees that the data on the target machine is synchronized to the latest version; it lacks historical data. If the source machine is attacked by ransomware, the target machine will also be compromised. This invention provides a method for continuously backing up Oracle logs, which allows Oracle data to be restored to a specified historical point in time.

[0006] The disadvantages of the three current methods are as follows:

[0007] The first method, which uses Oracle's built-in backup interface (RMAN Image Copy), can only back up data files and archived logs, not redo logs. When recovery is needed, only the backed-up data files and archived logs can be restored; without redo logs, some data will be lost. This method does not support redo log backups, resulting in data loss during recovery.

[0008] In the existing second method, when the Oracle application updates or inserts a large amount of data, the driver cannot process it in time, blocking Oracle operations for a long time, or even preventing continuous backups. It switches to bitmap mode, and when the data changes less frequently, it reads the corresponding sector content from the disk based on the bitmap content and saves it. During the period when the driver is in bitmap mode, the backup data is unavailable. Furthermore, if the developed driver has problems and is unstable, it may cause the production system to crash and shut down, leading to business interruption. Overall, this method impacts the performance of the user's production system and poses a threat to system stability. It affects the performance of the production machine, threatens the production system, and may result in unavailable data during periods of high IO pressure.

[0009] The third method requires an additional target machine with the Oracle application installed. Agent programs are installed on both the source and target machines. The source agent parses the source Oracle's logs to obtain SQL statements, which are then sent to the target agent. The target agent executes these SQL statements sequentially to operate on the target Oracle, achieving the goal of backing up the Oracle database. This approach only guarantees that the target data is synchronized to the latest version; it lacks historical data and cannot be rolled back. If the source machine is affected by ransomware, the target machine will also be subject to ransomware attacks. This method lacks historical data and requires an additional machine in advance, making it costly.

[0010] Therefore, it is necessary to develop a new method, system, device, and storage medium for continuous database log backup to solve the existing problems. Summary of the Invention

[0011] The purpose of this invention is to provide a method, system, apparatus, and storage medium for continuous database log backup, in order to solve the problem of the inability to continuously back up logs.

[0012] To achieve the above objectives, the present invention provides the following technical solution: a method for continuous database log backup, comprising:

[0013] Query the archived log list and copy all archived log files;

[0014] Copy all redo log files, generate redo log copy files, obtain the path of the redo log file currently being written and the offset of that file, read the current redo log content from the start position to the offset position, and overwrite the corresponding copy file to calibrate the content of the current redo log copy file;

[0015] Get the path to the redo log file currently being written and its offset. If the offset has changed, read the content between the previous offset and the current offset from the current redo log file and overwrite it in the corresponding replica file.

[0016] Preferably, the process involves obtaining the path and offset of the redo log file currently being written; if the log sequence number changes, a new archived log file is generated; this archived log file is copied; the remaining content of the previous redo log file is copied to the corresponding copy file; and the current redo log content is taken from the start position to the offset position and overwritten to the corresponding copy file.

[0017] Preferably, before the step of querying the archived log list, a full backup is performed through Oracle's backup interface to back up the data files.

[0018] Preferably, the querying of the archived log list includes: querying the archived log list using an Oracle SQL statement, wherein the SQL statement is select name from v$archived_log.

[0019] Preferably, the method for obtaining the path of the redo log file currently being written and the offset of that file includes:

[0020] select l.status,lf.member,k.CPODR_SEQ,k.CPODR_BNO,(k.CPODR_BNO+1)*512,k.CPODR_BOF from v$logl,v$logfilelf,x$kcccp k where l.group#=lf.group#and l.sequence#=k.CPODR_SEQ;

[0021] (k.CPODR_BNO+1)*512 represents the position where the redo log file will start writing next;

[0022] k.CPODR_SEQ is the log sequence number. If the log sequence number changes, the redo log file is switched, and the newly generated archived log is backed up.

[0023] The present invention also provides a database continuous log backup system, the system comprising:

[0024] The query module is used to query the list of archived logs;

[0025] The copy module is used to copy all archived log files;

[0026] The generation module is used to redo log copy files;

[0027] The get module is used to obtain the path of the redo log file currently being written and the offset of that file;

[0028] The read module is used to read the current redo log content from the start position to the offset position.

[0029] The present invention also provides a database continuous log backup device, comprising:

[0030] Memory for storing non-transitory computer-readable instructions; and

[0031] A processor for executing the computer-readable instructions such that, when executed by the processor, the computer-readable instructions implement the database continuous log backup method according to any one of the preceding claims.

[0032] The present invention also provides a storage medium for storing non-transitory computer-readable instructions, which, when executed by a computer, cause the computer to perform the database continuous log backup method described in any of the preceding claims.

[0033] The technical effects and advantages of this invention are as follows: The method, system, apparatus, and storage medium for continuous database log backup determine the currently being written redo log file and its offset position through relevant Oracle query statements. By calculating the file offset position, the content between the offset positions of the archive log file and the redo log file to be backed up is determined. This effect can be achieved by obtaining the file offset position directly or intermittently through other query commands. This facilitates the use of redo log files during recovery, reduces data loss during recovery, and eliminates the need for driver installation, thus having no performance impact on the system and applications, preventing system crashes, and eliminating the need for prior preparation of target machines, thereby reducing implementation costs. Furthermore, the currently being used redo log file and its corresponding file offset position can be obtained through Oracle's own views. The content of the corresponding redo log file is read based on the offset and written to the replica file, and this process is repeated cyclically to achieve continuous backup of the redo log file. Attached Figure Description

[0034] Figure 1 A flowchart illustrating an embodiment of the method of the present invention;

[0035] Figure 2 This is a diagram illustrating the copying method in steps one and two of the method embodiments of the present invention;

[0036] Figure 3 This is a schematic diagram illustrating the method in the embodiment of the present invention where, when the log sequence number remains unchanged and the redo log file has not been switched, the current offset position is obtained, the content between the previous offset and the current offset is read from the current redo log file, and the content is overwritten to the corresponding copy file.

[0037] Figure 4 This is a schematic diagram illustrating the process of copying portions of archived logs and redo log files from the original and backup locations when the sequence number changes in an embodiment of the method of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] This invention provides, for example Figure 1 The method for continuous log backup of a database shown includes:

[0040] Step 1: Use Oracle's SQL statements to query the archived log list and copy all archive log files, such as... Figure 2 As shown, the query statement for retrieving the list of archived logs is: select name from v$archived_log;

[0041] Step 2: Copy all redo log files to generate redo log replica files, one-to-one correspondence. Obtain the currently writing redo log file and its offset using an Oracle view. Read the current redo log content from the start position to the offset position and overwrite the corresponding replica file to calibrate the current redo log replica file content. Figure 2 As shown, the methods for querying the current active redo log file path and the offset position of the file currently being written include:

[0042] The query `select l.status,lf.member,k.CPODR_SEQ,k.CPODR_BNO,(k.CPODR_BNO+1)*512,k.CPODR_BOF from v$logl,v$logfilelf,x$kcccp k where l.group#=lf.group#and l.sequence#=k.CPODR_SEQ` is used to select the redo log entry. Here, `(k.CPODR_BNO+1)*512` represents the position where the redo log will start writing next.

[0043] k.CPODR_SEQ log sequence number. A change in the log sequence number indicates that the redo log has been switched, and the newly generated archived log needs to be backed up.

[0044] Step 3: Obtain the redo log file that Oracle is currently writing to and its offset through an Oracle view. If the log sequence number has not changed, it means Oracle is still writing the same redo log. If the offset position has not changed, there is no data change, and no copying is performed. If the offset position changes, read the content between the previous offset and the current offset from the current redo log file and overwrite it to the corresponding replica file. Figure 3 As shown, when Oracle has new data changes, but the redo log file hasn't been switched (i.e., the log sequence number hasn't changed), it only needs to copy the file content between the previous offset and the current offset from the current redo log; for example... Figure 4 As shown, when Oracle has new data changes and a redo log file switch occurs (i.e., the log sequence number changes), it is necessary to copy the new archive log file, the remaining content of the previous redo log file, and the content of the current redo log file from the beginning to the current offset.

[0045] Step 4: Repeat step 3 to achieve continuous log backup;

[0046] In step one, before querying the archived log list, a full backup is performed using Oracle's backup interface to back up the data files.

[0047] When data is inserted or updated in an Oracle database, the operation records are first saved to the redo log. The redo log generally consists of several files of a fixed size, and they are written to in a cyclical manner, overwriting existing files. For example, redo1.log, redo2.log, and redo3.log. Oracle will write to these files in the order of redo1.log, redo2.log, and redo3.log. When redo3.log is also full, it will switch to writing to redo1.log, and so on in a cyclical manner.

[0048] Archive logs are copied from redo logs and are archives of the redo logs. When redo3.log is full, writing to redo1.log is switched. Before overwriting redo1.log, the contents of redo1.log have been copied to archive log files such as archive15.log.

[0049] The present invention also provides a database continuous log backup system, the system comprising:

[0050] The query module is used to query the list of archived logs;

[0051] The copy module is used to copy all archived log files;

[0052] The generation module is used to redo log copy files;

[0053] The get module is used to obtain the path of the redo log file currently being written and the offset of that file;

[0054] The read module is used to read the current redo log content from the start position to the offset position.

[0055] The present invention also provides a database continuous log backup device, comprising:

[0056] Memory for storing non-transitory computer-readable instructions; and

[0057] A processor for executing the computer-readable instructions such that, when executed by the processor, the computer-readable instructions implement the database continuous log backup method according to any one of the preceding claims.

[0058] The present invention also provides a storage medium for storing non-transitory computer-readable instructions, which, when executed by a computer, cause the computer to perform the database continuous log backup method described in any of the preceding claims.

[0059] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0060] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for continuous log backup of a database, characterized in that: include: Query the archived log list and copy all archived log files; Copy all redo log files, generate redo log copy files, obtain the path of the redo log file currently being written and the offset of that file, read the current redo log content from the start position to the offset position, and overwrite the corresponding copy file to calibrate the content of the current redo log copy file; Get the path of the redo log file currently being written and the offset of that file. If the offset position changes, read the content between the previous offset and the current offset from the current redo log file and overwrite it to the corresponding replica file. The process involves obtaining the path and offset of the redo log file currently being written. If the log sequence number changes, a new archived log file is generated. This archived log file is copied, and the remaining content of the previous redo log file is copied to the corresponding copy file. The current redo log content is then taken from the start position to the offset position and overwritten to the corresponding copy file.

2. The method for continuous log backup of a database according to claim 1, characterized in that: Before the step of querying the archived log list, a full backup is performed using Oracle's backup interface to back up the data files.

3. The method for continuous log backup of a database according to claim 1, characterized in that: The query for the archived log list includes: querying the archived log list using an Oracle SQL statement, specifically the SQL statement `select namefromv$archived_log`.

4. The method for continuous log backup of a database according to claim 1, characterized in that: The method for obtaining the path of the redo log file currently being written and the offset of that file includes: select l.status,lf.member,k.CPODR_SEQ,k.CPODR_BNO,(k.CPODR_BNO+1)*512,k.CPODR_BOF from v$logl,v$logfilelf,x$kcccp k where l.group#=lf.group#andl.sequence#=k.CPODR_SEQ; (k.CPODR_BNO+1)*512 represents the position where the redo log will start writing next; k.CPODR_SEQ is the log sequence number. If the log sequence number changes, the redo log is switched, and the newly generated archived log is backed up.

5. A system for implementing the database continuous log backup method according to any one of claims 1-4, the system comprising: The query module is used to query the list of archived logs; The copy module is used to copy all archived log files; The generation module is used to redo log copy files; The get module is used to obtain the path of the redo log file currently being written and the offset of that file; The read module is used to read the current redo log content from the start position to the offset position.

6. A database continuous log backup device, comprising: Memory is used to store non-transitory computer-readable instructions; as well as A processor for executing the computer-readable instructions such that, when executed by the processor, the computer-readable instructions implement the database continuous log backup method according to any one of claims 1 to 4.

7. A storage medium for storing non-transitory computer-readable instructions that, when executed by a computer, cause the computer to perform the database continuous log backup method according to any one of claims 1 to 4.

Images