A database log archiving method, device, equipment and medium

Abstract

This invention relates to the field of data processing and discloses a database log archiving method, apparatus, device, and medium, comprising: identifying a target redo log file to be archived from the database; creating a preliminary redo log file with the same attributes as the target redo log file and empty content within the same logical disk as the target redo log file; moving the target redo log file to a preset archiving directory located within the same logical disk via MV operations; changing the filename and storage path of the preliminary redo log file to the original filename and storage path of the target redo log file via MV operations; and mounting the revised preliminary redo log file to the database. This invention solves the problems of high I / O consumption and low archiving efficiency in existing database log archiving technologies.

Description

Technical Field

[0001] This invention relates to the field of data processing, and in particular to a database log archiving method, apparatus, device, and medium. Background Technology

[0002] In database management systems, redo logs, as a core component for real-time recording of transaction operations, play a crucial role in ensuring data integrity and supporting crash recovery. When redo log files reach their capacity limit, their contents must be promptly converted into archive log files to prevent historical transaction records from being overwritten by new operations. Traditional archiving methods rely on a full file copy mechanism, requiring the system to read data block by block from the source redo log file into a buffer, perform integrity checks, and then write it to the target archive directory. This process consumes disk read and write operations equal to the file size, leading to direct competition between archiving tasks and mainline business operations for limited disk input / output resources during periods of high database workload. Especially during peak business periods, frequent archiving operations result in prolonged system response time, reduced transaction throughput, and may even pose service stability risks. Existing solutions, such as binary log subscription-based archiving methods, require additional database instance deployment and log conversion processing, introducing unnecessary resource overhead. While real-time block copy strategies shorten the duration of a single operation, they still cannot avoid the continuous I / O pressure from basic read and write operations, failing to fundamentally alleviate the disk resource contention problem.

[0003] Therefore, current technology still suffers from high archiving I / O consumption and low archiving efficiency. Summary of the Invention

[0004] This invention provides a database log archiving method, apparatus, computer equipment, and storage medium to solve the problems of high I / O consumption and low archiving efficiency in the prior art.

[0005] Firstly, a database log archiving method is provided, including: Identify the target redo log files that the database needs to archive; On the same logical disk as the target redo log file, create a pre-redo log file with the same attributes as the target redo log file and empty content; The target redo log file is moved to a preset archive directory located on the same logical disk using the MV operation. The MV operation is used to change the file name and storage path of the prepared redo log file to the original file name and storage path of the target redo log file. Mount the prepared redo log file, after changing its filename and storage path, to the database.

[0006] Optionally, create a preliminary redo log file with the same attributes as the target redo log file and empty content, including: Create a pre-redo log file that is the same size and format as the target redo log file but has no content. Write the unique identifier information of the target redo log into the header of the prepared redo log file; Before mounting the prepared redo log file (with changed filename and storage path) to the database, the process also includes: When the target redo log file is successfully moved to the preset archive directory, the file header of the prepared redo log file is reinitialized.

[0007] Optionally, after moving the target redo log file to a preset archive directory located on the same logical disk via MV operation, the method further includes: Change the name of the target redo log file according to the naming rules corresponding to the preset archive directory.

[0008] Optionally, create a preliminary redo log file with the same attributes as the target redo log file and empty content, including: When a target redo log file that needs to be archived is identified, a file with the same attributes as the target redo log file and empty content is selected from the prepared file pool as a prepared redo log file. The prepared file pool is located on the same logical disk as the target redo log file, and contains multiple files with the same attributes as the target redo log file but empty content.

[0009] Optionally, creating a preliminary redo log file with the same attributes as the target redo log file but empty content, within the same logical disk as the target redo log file, further includes: Create a preliminary redo log file with the same attributes as the target redo log file and empty content, located on the same logical disk and in the same storage path as the target redo log file.

[0010] Optionally, after creating a preliminary redo log file with the same attributes as the target redo log file and empty content, the process further includes: Establish a mapping relationship between the prepared redo log file and the target redo log; After changing the filename and storage path of the prepared redo log file to the original filename and storage path of the target redo log file through the MV operation, the method further includes: Based on the mapping relationship, verify whether the archived target redo log file and the prepared redo log file after changing the file name and storage path correspond; If the verification passes, the prepared redo log file with the changed file name and storage path will be mounted to the database; If the verification fails, the target redo log file that needs to be archived in the database will be re-identified.

[0011] Optionally, after mounting the prepared redo log file (with modified filename and storage path) to the database, the method further includes: Clear the mapping relationship between the prepared redo log file and the target redo log.

[0012] Secondly, a database log archiving device is provided, including: The identification module is used to identify the target redo log files that the database needs to archive; A creation module is used to create a preliminary redo log file with the same attributes as the target redo log file and empty content, on the same logical disk as the target redo log file; The archiving module is used to move the target redo log file to a preset archive directory located on the same logical disk via MV operation; The module is used to change the filename and storage path of the prepared redo log file to the original filename and storage path of the target redo log file through MV operation; The mounting module is used to mount the prepared redo log file, after changing its file name and storage path, to the database.

[0013] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described database log archiving method.

[0014] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described database log archiving method.

[0015] This application provides a database log archiving method, apparatus, device, and medium, comprising: identifying a target redo log file that needs to be archived in the database; creating a pre-record redo log file with the same attributes as the target redo log file and empty content within the same logical disk as the target redo log file; moving the target redo log file to a preset archiving directory located within the same logical disk via MV operation; changing the filename and storage path of the pre-record redo log file to the original filename and storage path of the target redo log file via MV operation; and mounting the pre-record redo log file with the changed filename and storage path to the database. When a target redo log file is identified as needing archiving, an empty preparation file with the same attributes as the target file is first created on the same logical disk. This creation operation only involves the creation of file system metadata and a small amount of space allocation, with I / O overhead far lower than writing all data. The target redo log file is then moved to the archive directory using an MV (Move-In, Move-Out) operation. Since the target redo log file and the preset archive directory are located on the same logical disk, this MV operation only modifies the file system metadata, avoiding actual data block reads and writes, thus significantly reducing disk I / O. Finally, the preparation redo log file is renamed and moved to the original target redo log file's location using an MV operation, allowing it to take over the original file's functionality. This step also utilizes the metadata modification feature of the MV operation, avoiding data copying. Therefore, this scheme effectively solves the problems of excessive disk I / O consumption and low archiving efficiency in traditional schemes. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a database log archiving method provided in an embodiment of the present invention; Figure 2 This is another flowchart illustrating the database log archiving method provided in this embodiment of the invention; Figure 3 This is a schematic diagram of a database log archiving device provided in an embodiment of the present invention.

[0018] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," or "in response to determination." Similarly, the phrase "if determined" or "if matched to [described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once matched to [described condition or event]," or "in response to matched to [described condition or event]."

[0023] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0026] The data processing methods, apparatus, devices, and media provided in the embodiments of this application will be described in detail below.

[0027] like Figure 1 The diagram shown is a flowchart illustrating a database log archiving method provided in an embodiment of this application. Specifically, the database log archiving method may include the following steps: S101. Identify the target redo log files that the database needs to archive.

[0028] A database is a system used to store, manage, and retrieve data. In this system, data is organized in a structured manner and accessed and manipulated through specific interfaces.

[0029] Log archiving refers to the process of backing up and permanently storing the transaction logs recorded in the database. This method ensures that the database can be restored to a consistent state in the event of a failure and provides the ability to audit historical data.

[0030] The target redo log file, also known as the redo log file, is a critical file that records all transaction operations in real time. These files document all changes made to the database and are essential for crash recovery and ensuring data integrity. When a file reaches a certain threshold, it needs to be archived.

[0031] This identification process can be implemented in several ways. For example, the database system can periodically check the size of each redo log file and mark it as needing to be archived when the file size reaches a preset threshold. Alternatively, it can be identified as archiveable once all transactions in a log file have been committed and written to persistent storage, based on the progress of the log sequence number. In another implementation, the database administrator can manually trigger the archiving operation, specifying a particular redo log file to be archived.

[0032] S102. On the same logical disk as the target redo log file, create a backup redo log file with the same attributes as the target redo log file and empty content.

[0033] A logical disk refers to a storage unit recognized and managed at the operating system or file system level. A physical disk can be divided into multiple logical disks, or multiple physical disks can be combined into a single logical disk. The operations in this method are performed within the same logical disk to optimize file operation efficiency.

[0034] A prepared redo log file is a file created on the same logical disk as the target redo log file and is empty. It has the same attributes as the target redo log file but contains no content. This file serves as a replacement for the target redo log file after it has been archived, maintaining the continuity of the database redo logs.

[0035] This creation operation provides a placeholder for the target redo log file that will soon be archived. Specifically, it calls the operating system's file creation interface, specifying that the new file's size, file type, permissions, and other attributes are consistent with the target redo log file, and ensuring that the newly created file is empty. For example, if the target redo log file is 1GB in size and in binary log file format, the created pre-built redo log file will also be a 1GB empty binary log file. This creation process typically involves allocating storage space on the file system and creating file metadata.

[0036] S103. Move the target redo log file to the preset archive directory using the MV operation.

[0037] MV operations refer to file move commands provided by the operating system. When the source file and the target location are located on the same logical disk, this operation typically only involves modifying file system metadata rather than copying actual data blocks, thus offering high execution efficiency and low resource consumption.

[0038] A default archive directory refers to a specific storage location used to store archived redo log files. This directory is typically configured on storage media accessible to the database system and follows specific naming and organization rules.

[0039] This MV operation leverages operating system file system features. When the source file and target directory reside on the same logical disk, the file move operation typically only modifies metadata within the file system, such as file paths and names, without actually copying the file content. This significantly reduces disk I / O resource consumption. For example, a redo log file named "redo_log_01.log," after being identified as needing archiving, can be moved from the current online log directory to an archive directory named "archive_logs" via an MV operation.

[0040] S104. Change the filename and storage path of the prepared redo log file to the filename and storage path of the original target redo log file through the MV operation.

[0041] The purpose of this step is to have the newly created empty file take over the location and identity of the original target redo log file. Similarly, since this operation occurs within the same logical disk, it primarily involves updating file system metadata rather than moving data content. For example, if the previously created prepare redo log file was named "prepare_redo_01.log", and the original target redo log file's path and filename were " / data / redo / redo_log_01.log", then the MV operation will rename "prepare_redo_01.log" and move it to " / data / redo / redo_log_01.log".

[0042] S105. Mount the prepared redo log file with the changed file name and storage path to the database.

[0043] Mounting refers to connecting a file system or storage device to a point in the operating system's directory tree, making its contents accessible and usable by the database system. Mounting a pre-prepared redo log file to the database means that the database recognizes the file and uses it as a new redo log file.

[0044] This mount operation enables the database system to recognize and begin using the new, empty redo log file. The database system updates its internal metadata, registering the file as one of the currently available redo log files and preparing it to receive new transaction log writes. For example, the database management system executes an internal command to add the new file at the path " / data / redo / redo_log_01.log" to its redo log group, making it an active log file.

[0045] The following example will provide a more detailed explanation of the above technical solution: like Figure 2 As shown, the database is configured with three redo log files: REDO1, REDO2, and REDO3, each 1GB in size. These files are all stored on the same logical disk. The database system operates normally and continuously writes transaction logs to these files.

[0046] When the transaction log records in the REDO1 file reach their capacity limit, or when other preset archiving conditions are met, the database's archiving thread will recognize that the REDO1 file needs to be archived. At this time, REDO1 is marked as pending archiving.

[0047] Next, on the same logical disk where the REDO1 file resides, the archive thread creates a new file, for example, named REDO_PREPARE. This REDO_PREPARE file has the same size and file format as the REDO1 file, and its content is empty. This creation operation only involves allocating space in the file system and creating file metadata, consuming fewer disk I / O resources compared to writing a full dataset.

[0048] Subsequently, the database system unloads the REDO1 file from its redo log group, preventing it from being used by the database to write new transaction logs. Then, through an MV operation, the REDO1 file is moved from its original storage path, such as " / data / redo / REDO1", to a predefined archive directory. Since the REDO1 file and the archive directory reside on the same logical disk, this MV operation only modifies metadata in the file system, such as file paths and names, without performing actual data block copying, thus significantly reducing disk I / O overhead.

[0049] After the REDO1 file is successfully moved, the archive thread performs another MV operation. This operation renames the previously created REDO_PREPARE file (e.g., located at " / data / redo / REDO_PREPARE") and moves it to the original storage path of the REDO1 file (i.e., " / data / redo / REDO1"). Through this operation, the REDO_PREPARE file effectively takes over the location and identity of the original REDO1 file, becoming the new redo log file. This MV operation also only involves metadata modification and is highly efficient.

[0050] Finally, the new, empty REDO1 file (renamed from the REDO_PREPARE file) is remounted to the database. The database system updates its internal configuration to recognize this file as a usable redo log file and prepares to receive subsequent transaction log writes. At this point, the REDO1 file archiving process is complete, and the database's redo log functionality can run continuously without waiting for time-consuming data replication operations. Subsequent REDO2 and REDO3 files will also follow the same process when they meet the archiving criteria.

[0051] Based on the above examples, the database log archiving method provided in this embodiment demonstrates significant technical contributions. Traditional database log archiving schemes typically employ file copying for backup. This means that all data needs to be read from the source redo log file and then written to the target archive file. For example, in the above example, if the REDO1 file size is 1GB, the traditional copy operation would involve 1GB of disk read I / O and 1GB of disk write I / O. When the database workload is large, this large amount of disk I / O operations will compete with the main database operations for valuable system resources, thereby affecting the overall operating efficiency and external service performance of the database.

[0052] This application provides a database log archiving method, comprising: identifying a target redo log file that needs to be archived in the database; creating a preparatory redo log file with the same attributes as the target redo log file and empty content within the same logical disk as the target redo log file; moving the target redo log file to a preset archiving directory located within the same logical disk via MV operation; changing the filename and storage path of the preparatory redo log file to the original filename and storage path of the target redo log file via MV operation; and mounting the preparatory redo log file with the changed filename and storage path to the database. When a target redo log file is identified as needing archiving, an empty preparation file with the same attributes as the target file is first created on the same logical disk. This creation operation only involves the creation of file system metadata and a small amount of space allocation, with I / O overhead far lower than writing all data. The target redo log file is then moved to the archive directory using an MV (Move-In, Move-Out) operation. Since the target redo log file and the preset archive directory are located on the same logical disk, this MV operation only modifies the file system metadata, avoiding actual data block reads and writes, thus significantly reducing disk I / O. Finally, the preparation redo log file is renamed and moved to the original target redo log file's location using an MV operation, allowing it to take over the original file's functionality. This step also utilizes the metadata modification feature of the MV operation, avoiding data copying. Therefore, this scheme effectively solves the problems of excessive disk I / O consumption and low archiving efficiency in traditional schemes.

[0053] In some embodiments described above, a method is proposed to identify the target redo log file that needs to be archived in the database, and to create a preparatory redo log file with the same attributes as the target redo log file but empty content, on the same logical disk as the target redo log file. Then, the target redo log file is moved to a preset archive directory using an MV operation, and the filename and storage path of the preparatory redo log file are changed to the original filename and storage path of the target redo log file. Finally, the modified preparatory redo log file is mounted to the database. However, in practice, simply creating an empty file with the same attributes may not adequately guarantee the logical association and data consistency between the preparatory redo log file and the original target redo log file, especially during system anomalies or fault recovery, potentially leading to confusion in database log management or data integrity risks.

[0054] In response, this application further proposes to create a preparatory redo log file with the same attributes as the target redo log file and empty content, including: creating a preparatory redo log file with the same size and format as the target redo log file and empty content; writing the unique identifier information of the target redo log into the header of the preparatory redo log file; before mounting the preparatory redo log file with the changed file name and storage path to the database, it also includes: re-initializing the header of the preparatory redo log file when the target redo log file is successfully moved to the preset archive directory.

[0055] Specifically, to ensure that the prepared redo log file can seamlessly replace the original target redo log file, the system accurately obtains the size and format information of the target redo log file, such as the file size in bytes, internal block structure, and metadata layout, and creates a new file with the same physical and logical structure accordingly. This can be done by querying the file system's metadata or by using APIs provided by the database management system.

[0056] In addition, a unique identifier for the target redo log file is written into the header of the prepared redo log file. This unique identifier can be the target redo log file's sequence number, timestamp, version number, or a combination thereof, used to establish a clear association between the prepared redo log file and the archived target redo log file at the logical level. This information is typically written to a specific area in the file header so that the database system can identify and verify it during subsequent processing.

[0057] Furthermore, this application also includes reinitializing the file header of the prepared redo log file upon successful movement of the target redo log file to the preset archive directory. This step ensures that the prepared redo log file is in a clean and consistent initial state before being mounted to the database. Reinitialization may include operations such as clearing old control information in the file header, writing a new start sequence number, or resetting internal pointers to ensure that the file can be correctly recognized and used by the database as a new redo log file.

[0058] The above method effectively improves the reliability and data integrity of database log archiving by refining the creation and management process of the pre-redo log file. First, by creating a pre-redo log file with the exact same size and format as the target redo log file, it ensures that the new file maintains physical structural consistency with the original file. This eliminates the need for additional adaptation or conversion when switching log files, thus guaranteeing smooth operation. Second, by writing the unique identifier of the target redo log file into the header of the pre-redo log file, a logical association is established between the old and new log files. This not only facilitates verification during archiving but, more importantly, allows the database to accurately identify and match the relationship between the archived log file and the current log file based on this unique identifier when a system failure occurs or recovery is required. This avoids log chain breaks or data inconsistencies that may result from file replacement. Finally, only after the target redo log file has been successfully moved to the preset archive directory is the header of the prepared redo log file reinitialized. This sequence ensures that the new prepared redo log file is only officially ready after the original redo log file has been safely archived, avoiding an uncertain state for the prepared redo log file in the event of archiving failure or interruption, thus enhancing the robustness of the entire archiving process. Through the synergistic effect of these steps, the solution in this application significantly improves the security and recoverability of archiving operations while maintaining the continuity of database logs.

[0059] The following is a concrete example. Suppose a database system needs to archive a target redo log file named `redo_log_01.rdo`. First, the system reads the metadata of `redo_log_01.rdo`, determines its size to be 512MB, and uses a specific database log file format. Then, the system creates a new empty file, for example, named `temp_redo_log_01.rdo`, on the same logical disk as `redo_log_01.rdo`, ensuring its size is also 512MB and its internal structure conforms to the database log file format. Next, the system extracts a unique log sequence number (e.g., LSN 123456) from `redo_log_01.rdo` and writes this LSN 123456 into the header of `temp_redo_log_01.rdo`. After completing these preparations, the system performs an MV operation to move `redo_log_01.rdo` to the preset archive directory. Once the move operation completes successfully, the system immediately reinitializes the header of `temp_redo_log_01.rdo`, for example, by clearing its old control information and writing an initial status flag indicating that it is a new redo log file. Finally, `temp_redo_log_01.rdo` is renamed to `redo_log_01.rdo` and mounted to the database to continue being used as a redo log file.

[0060] Through the above technical solution, this application effectively solves the problems of insufficient logical correlation and data consistency risks that may exist between the prepared redo log file and the original target redo log file during the database log archiving process. By precisely matching the file size and format, it ensures that the prepared redo log file can be seamlessly accepted and used by the database system. By writing unique identification information in the file header, clear logical traceability is provided for the replacement of log files, greatly enhancing the integrity and traceability of the log chain. In addition, setting the file header reinitialization time after the target redo log file has been successfully moved avoids the prepared file being in an uncertain state due to the interruption of the archiving operation, thereby significantly improving the robustness of the entire log archiving process. This enables the database to operate more stably and securely during log archiving, reduces the risk of data loss or corruption, and provides a more solid foundation for system failure recovery.

[0061] In one embodiment, this application further proposes that after moving the target redo log file to a preset archive directory through MV operation, the name of the target redo log file is changed according to the naming rules corresponding to the preset archive directory.

[0062] The naming rules corresponding to the preset archive directory refer to a set of naming conventions or strategies pre-defined for files in the archive directory, aiming to ensure the uniformity, identifiability, and manageability of archived files. These rules can be set based on various dimensions. For example, file names can be required to include the archive date and timestamp for chronological retrieval; or, file names can be required to include the database identifier and log sequence number for tracing the log source and content. Changing the name of the target redo log file refers to renaming the target redo log file that has been moved to the preset archive directory so that its new name conforms to the naming rules of the preset archive directory. This operation can be achieved by calling the file system interface provided by the operating system, or by using APIs or scripts provided by the database management system or dedicated archive management tools.

[0063] After moving the target redo log file to the preset archive directory via MV operations, this application further introduces a step of renaming the target redo log file according to the naming rules corresponding to the preset archive directory. This operation ensures that the archived log file immediately conforms to the unified management and retrieval standards of the archive storage area after entering the area. By renaming the file after it has been moved, errors or inconsistencies that may occur during the move due to filenames not conforming to the target directory specifications can be avoided. This ensures that the archived files are not only physically located in the correct storage location, but are also logically organized effectively through standardized naming, thus seamlessly integrating with the database's archiving strategy and improving the efficiency and maintainability of the entire archiving system.

[0064] As a specific implementation, once the database identifies a target redo log file that needs to be archived and moves it to a preset archive directory via MV operations, the system can immediately rename the file. For example, the preset archive directory might require all archived redo log files to include the archive date and log sequence number. Suppose a target redo log file is named "redo01.log", its log sequence number is 12345, and its archive date is October 26, 2023. After the file is moved, the system will rename "redo01.log" to "20231026_12345_redo01.log" according to a preset naming rule (e.g., "YYYYMMDD_SEQ_FILENAME.log"). This renaming operation can be achieved by calling the file system interface provided by the operating system or the API of a specific archive management tool.

[0065] By renaming the target redo log file according to the naming rules of the preset archive directory immediately after it has been moved, this application effectively solves the problem of low management and retrieval efficiency caused by non-standard archive file naming. This ensures that all archived log files have a unified and clear identifier after entering the archive system, greatly improving the identifiability, manageability, and retrieval of archived files, thereby optimizing the long-term storage and maintenance efficiency of database logs.

[0066] In some of the embodiments described above in this application, the database log archiving method needs to create a new preparatory redo log file on the same logical disk as the target redo log file each time archiving is required. However, frequently creating new files introduces additional system overhead, including file system metadata operations and disk space allocation. This may lead to reduced efficiency of the archiving process and increased response latency of the database system, especially in high-concurrency or large-data-volume scenarios, which adversely affects system performance.

[0067] In response, this application further proposes the above-mentioned creation of an empty preparatory redo log file with the same attributes as the target redo log file, including: when a target redo log file that needs to be archived is identified, selecting a file with the same attributes as the target redo log file and empty content from the preparatory file pool as the preparatory redo log file; the preparatory file pool is located on the same logical disk as the target redo log file and contains multiple files with the same attributes as the target redo log file and empty content.

[0068] The "prepared file pool" refers to a set of pre-prepared empty files that have the same attributes as the target redo log file, such as file size and format. When the system identifies a target redo log file that needs to be archived, it no longer creates a new file on the fly. Instead, it quickly selects a file from this pre-existing prepared file pool that meets the criteria. This selection can be done by finding the first available empty file in the pool or by selecting a file based on a strategy such as round-robin or load balancing. This avoids the overhead of creating files in real time, thus improving archiving efficiency. Furthermore, the fact that "the prepared file pool and the target redo log file are on the same logical disk" ensures that subsequent MV operations can be completed within the same file system or storage volume, avoiding cross-disk data copying and significantly improving operational efficiency. If the files are on different logical disks, the MV operation would essentially become a copy-delete operation, which would take longer. The presence of multiple empty files matching the target redo log file attributes ensures the sufficiency of the prepared file pool. Even during multiple consecutive archiving operations, a continuous supply of qualified prepared redo log files is provided, preventing the need to temporarily create files due to pool exhaustion and maintaining the smoothness and efficiency of the archiving process. These empty files can be created once during system initialization or dynamically replenished when the number of files in the file pool falls below a certain threshold.

[0069] This application's solution optimizes the creation process of pre-redo log files in the aforementioned database log archiving method by introducing a pre-file pool mechanism. Specifically, when the database identifies a target redo log file to be archived, it no longer performs time-consuming file creation operations. Instead, it directly selects an empty file with attributes consistent with the target redo log file from a pre-maintained pre-file pool as the pre-redo log file. This pre-file pool is designed to reside on the same logical disk as the target redo log file, enabling subsequent MV operations to be completed efficiently, as the file system only needs to update metadata without actually moving data blocks. Furthermore, the pre-file pool contains multiple empty files matching the attributes, ensuring that in continuous archiving scenarios, the system can always quickly obtain a usable pre-redo log file, thus avoiding the decline in archiving efficiency caused by file creation delays. This mechanism transforms the pre-redo log file preparation phase from dynamic creation to rapid selection, significantly reducing the overhead of file system operations, ensuring the continuity and high performance of the database log archiving process, and effectively solving the efficiency problem caused by frequent file creation.

[0070] The following is a concrete example. Suppose a database system needs to archive a target redo log file named `redo01.log`, which is 1GB in size and in the Oracle database redo log format. When implementing this solution, the system will pre-create a pool of prepared files on the logical disk where `redo01.log` is located, such as ` / u01 / oradata / `. This pool can contain multiple empty files, such as `temp_redo_01.log` and `temp_redo_02.log`, each 1GB in size and with the same format as `redo01.log`. When the database recognizes that `redo01.log` needs to be archived, the system will select an empty file from this pool, such as `temp_redo_01.log`, as the prepared redo log file. Subsequently, the `redo01.log` is moved to the preset archive directory through MV operation, and the file name and storage path of `temp_redo_01.log` are changed to the original file name and storage path of `redo01.log`. Finally, it is mounted to the database.

[0071] The above technical solution transforms the creation of pre-requisite redo log files from real-time creation to selection from a pre-prepared pool of pre-requisite files. Since the files in the pool have pre-allocated space and are initialized, and reside on the same logical disk as the target redo log file, the system can quickly obtain a suitable pre-requisite file when archiving is needed, avoiding the real-time overhead of file system metadata operations and disk space allocation. This significantly improves the efficiency of database log archiving and reduces the impact of the archiving process on database system performance. Especially in high-frequency, high-volume log archiving scenarios, it effectively reduces archiving latency, ensuring the continuity and stability of database services.

[0072] In some of the embodiments described above in this application, a preliminary redo log file is proposed to be created on the same logical disk as the target redo log file. However, in practice, if the preliminary redo log file and the target redo log file are not on the same storage path, it may increase the complexity of file system operations.

[0073] To address this, this application further proposes creating a preparatory redo log file with identical attributes and empty content, located on the same logical disk and in the same storage path as the target redo log file. "On the same logical disk and in the same storage path as the target redo log file" means that the created preparatory redo log file and the target redo log file not only reside on the same logical disk but also occupy the same level and position in the file system's directory structure. This positioning method ensures that the file system can process subsequent file operations in the most efficient manner. Specifically, the database management system can first obtain the complete storage path of the target redo log file and then generate the preparatory redo log file based on this path. "Creating a preparatory redo log file with identical attributes and empty content" means that the newly created file is consistent with the target redo log file in terms of size, format, and access permissions, but it does not contain any actual log data. This can be achieved by pre-allocating disk space of the same size as the target redo log file when creating the file and initializing its content to zero or a specific null value.

[0074] The proposed solution involves identifying the target redo log file to be archived in the database, and then creating a blank backup redo log file with identical attributes on the same logical disk and in the same storage path as the target redo log file. When the target redo log file is moved to the preset archive directory via an MV operation, since the backup redo log file already exists in the same storage path as the original target redo log file, subsequent MV operations to change the filename and storage path of the backup redo log file to the original target redo log file only require the file system to perform a single directory entry renaming operation. This process avoids cross-path file movement, significantly shortens the file replacement time window, reduces the potential risks to the database during log file switching, and improves the efficiency and reliability of the entire archiving operation.

[0075] The following is a concrete example. Assume the target redo log file currently being used by the database is ` / u01 / app / oracle / oradata / mydb / redo01.log`. When this file needs to be archived, the solution in this application will create a preliminary redo log file named `temp_redo.log` in the storage path ` / u01 / app / oracle / oradata / mydb / `. This `temp_redo.log` file will have the same file size, block size, and file permissions as the `redo01.log` file, but its internal data area will be empty. Subsequently, after `redo01.log` is successfully moved to an archive directory such as ` / archive / mydb / `, the system will immediately rename `temp_redo.log` to `redo01.log`, allowing the database to continue writing new redo log information to that path.

[0076] By creating the proposed redo log file on the same logical disk and storage path as the target redo log file, the file replacement process is greatly simplified. Since renaming operations within the same path in the file system are typically atomic, this significantly reduces the downtime during database log file switching, lowering the risk of database I / O latency or potential log file unavailability caused by file replacement operations. Furthermore, this approach reduces complex operations within the file system, improving the overall efficiency and stability of archiving operations and ensuring a smooth database log archiving process.

[0077] In some of the above implementations, a method for archiving database logs by creating a pre-redo log file and performing MV operations is proposed. However, in actual operation, due to system environment, concurrent operations, or other unforeseen factors, there may be situations where the archived target redo log file and the replaced pre-redo log file do not correspond correctly. This may lead to damage to the integrity of the database log chain, thereby affecting the normal operation of the database and data consistency.

[0078] To address this, this application further proposes, after creating a preparatory redo log file with the same attributes as the target online redo log file and empty content, establishing a mapping relationship between the preparatory redo log file and the target online redo log; after changing the filename and storage path of the preparatory redo log file to the original filename and storage path of the target redo log file through MV operation, verifying whether the archived target redo log file and the preparatory redo log file with the changed filename and storage path correspond according to the mapping relationship; if the verification passes, mounting the preparatory redo log file with the changed filename and storage path to the database; if the verification fails, re-identifying the target redo log file that needs to be archived in the database.

[0079] Establishing the mapping relationship between the prepared redo log file and the target online redo log refers to the system recording which prepared redo log file will replace which target redo log file before the database log archiving operation begins. This mapping relationship is the basis for subsequent verification operations, ensuring the logical correctness of the replacement operation. This mapping relationship can be maintained in the database's memory structure using a temporary data structure, such as a hash table or linked list, to store the association between the unique identifier of the prepared redo log file (such as file path, inode number) and the unique identifier of the target redo log (such as log sequence number LSN, file path). Alternatively, this mapping relationship can be persistently stored in a temporary file or a specific area of ​​the database's control file so that verification can still be performed after system restart or fault recovery.

[0080] The core verification step for confirming the correctness of the archiving operation is to verify whether the archived target redo log file and the prepared redo log file with changed filenames and storage paths correspond, based on the mapping relationship. It utilizes the previously established mapping relationship to check whether the actual operation results meet expectations. Specifically, after the MV operation is completed, the system can read the metadata (such as filename, size, checksum) of the target redo log file and the metadata of the prepared redo log file in the archive directory, and compare them with the information recorded in the mapping relationship. For example, it checks whether the new filename and path of the prepared redo log file are completely consistent with the original filename and path of the target redo log file, and whether the archived target redo log file actually exists in the preset archive directory. Additionally, it can also confirm whether the prepared redo log file has successfully "occupied" the position of the target redo log file and whether the original target redo log file has been successfully moved to the archive directory by comparing the inode information or file handles at the file system level.

[0081] If the verification passes, the prepared redo log file, with its filename and storage path changed, is mounted to the database. This refers to the step of officially putting the prepared redo log file into use after confirming that the file operations are correct. The database management system executes internal commands or API calls to register the file with the database's log management module, making it one of the currently available redo log files. This typically involves updating the database's control file and memory structure, informing the database of the file's existence and its position in the log chain.

[0082] If the verification fails, the system re-identifies the target redo log file that needs to be archived in the database. This is an error handling mechanism provided by the solution to ensure that the system can roll back or retry when the archiving operation fails or is inconsistent, avoiding data loss or corruption. The system can trigger an alarm or error log, roll back the current operation, and then restart the entire archiving process, starting from scratch to identify the target redo log file that needs to be archived. Alternatively, if the target redo log file is found to be incorrectly archived, it can be moved again. If the repair fails, it rolls back and re-identifies the file.

[0083] The following is a concrete example. Assume a database system is running and needs to archive a target redo log file named `redo01.log`. First, before performing the MV operation, the database system maintains a temporary mapping internally, for example, recording in a hash table in memory: `{" / tmp / prep_redo01.log":" / path / to / redo01.log"}`, indicating that a newly created empty file ` / tmp / prep_redo01.log` will replace ` / path / to / redo01.log`. Next, the system performs the MV operation, moving the original `redo01.log` to the archive directory. Subsequently, the pre-emptive redo log file ` / tmp / prep_redo01.log` is renamed and moved to the original location of `redo01.log`. After these MV operations are completed, the system performs a check based on the previously established mapping. The check process may include checking whether ` / path / to / redo01.log` has been occupied by the expected empty pre-emptive redo log file. If the verification passes, confirming that all file operations have completed as expected, the database log writing process will be notified that ` / path / to / redo01.log` is ready to be used as a new redo log file and will update the database control file to reflect this status. Conversely, if the verification fails—for example, if the second `mv` command fails to execute successfully, resulting in ` / path / to / redo01.log` still being missing or containing old data—the system will log an error and restart the entire archiving process, starting from scratch to identify `redo01.log` or another suitable log file for archiving, thus preventing the database from attempting to write to a non-existent or incorrect log file.

[0084] By introducing a mapping relationship and a verification mechanism during the database log archiving process, this application effectively solves the problem of log file replacement errors caused by the uncertainty of file operations. This ensures that the archived target redo log file and the replaced target redo log file can correctly correspond before the prepared redo log file is mounted to the database, significantly improving data consistency and system stability during log archiving operations. When verification fails, the system can promptly detect and re-identify the target redo log file that needs to be archived, providing an effective error recovery and adaptive mechanism, further enhancing the reliability of the database system.

[0085] In one embodiment, this application further proposes that after mounting the prepared redo log file with the changed file name and storage path to the database, the mapping relationship between the prepared redo log file and the target online redo log is cleared.

[0086] The purpose of clearing the mapping between the preparatory redo log file and the target online redo log is to free up system resources and simplify subsequent management. Once the preparatory redo log file is successfully mounted and assumes the function of the original target redo log file, the mapping previously established for verification purposes has completed its task. Clearing this mapping prevents it from continuing to occupy memory, file handles, or other system resources, and prevents the system from misusing or maintaining an invalid mapping in subsequent operations. Specifically, this clearing operation can be achieved by searching and deleting metadata records associated with the mapping in the database management system's memory. For example, if the mapping is stored in a hash table or linked list, the corresponding entry can be located and removed using a key value such as the unique identifier of the target online redo log. Alternatively, any file system-level links or metadata associated with the mapping can be unlinked or deleted by calling APIs provided by the operating system or file system. For example, if the mapping is implemented through soft links or hard links, these links can be deleted after successful mounting.

[0087] The following concrete example illustrates this: When a database system needs to archive a target redo log file named `redo01.log`, the system creates a preliminary redo log file named `temp_redo.log` on the same logical disk and establishes an internal mapping relationship between `temp_redo.log` and `redo01.log`, for example, recording `{redo01.log -> temp_redo.log}` in the database's memory structure. After `redo01.log` is moved to the archive directory, `temp_redo.log` is renamed to `redo01.log` and mounted to the database. After successful mounting, the system can execute an internal command or call a specific API to find and delete the mapping relationship between `redo01.log` and `temp_redo.log` maintained internally by the database.

[0088] By employing the above technical solution, after the preparatory redo log file is successfully mounted to the database and replaces the original target online redo log file, the mapping relationships established for verification purposes are promptly cleared. This effectively prevents invalid mapping relationships from continuously consuming system resources, such as memory or metadata storage space, thereby optimizing system resource utilization. Simultaneously, by removing unnecessary mapping information, the complexity and potential errors caused by processing or querying redundant mappings in subsequent database operations are reduced, improving system operating efficiency and stability.

[0089] In one embodiment, this application also provides a database log archiving apparatus. For example... Figure 3As shown, it includes an identification module 31, a creation module 32, an archiving module 33, a modification module 34, and a mounting module 35. Detailed descriptions of each functional module are as follows: Identification module 31 is used to identify the target redo log files that need to be archived in the database; The creation module 32 is used to create a preliminary redo log file with the same attributes as the target redo log file and empty content, on the same logical disk as the target redo log file. The archiving module 33 is used to move the target redo log file to a preset archiving directory through MV operation; Modify module 34, used to change the file name and storage path of the prepared redo log file to the original file name and storage path of the target redo log file through MV operation; Mounting module 35 is used to mount the prepared redo log file with changed file name and storage path to the database.

[0090] This invention also provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the aforementioned database log archiving method; to avoid repetition, this will not be described again here. Alternatively, the electronic device can implement the functions of each module in this embodiment of the database log archiving device; this will also not be described again here.

[0091] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for network communication. When executed by the processor, the computer program implements the functions or steps of a database log archiving method.

[0092] This invention also provides a readable storage medium storing a program that, when executed by a processor, implements the aforementioned database log archiving method. To avoid repetition, this will not be described further here. Alternatively, when the program is executed by a processor, it implements the functions of each module in this embodiment of the database log archiving device; this will also not be described further here.

[0093] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

[0094] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0095] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A database log archiving method, characterized in that, include: Identify the target redo log files that the database needs to archive; On the same logical disk as the target redo log file, create a pre-redo log file with the same attributes as the target redo log file and empty content; The target redo log file is moved to a preset archive directory located on the same logical disk using the MV operation. The MV operation is used to change the file name and storage path of the prepared redo log file to the original file name and storage path of the target redo log file. Mount the prepared redo log file, after changing its filename and storage path, to the database.

2. The database log archiving method according to claim 1, characterized in that, Creating a preliminary redo log file with the same attributes as the target redo log file and empty content includes: Create a pre-redo log file that is the same size and format as the target redo log file but has no content. Write the unique identifier information of the target redo log into the header of the prepared redo log file; Before mounting the prepared redo log file (with changed filename and storage path) to the database, the process also includes: When the target redo log file is successfully moved to the preset archive directory, the file header of the prepared redo log file is reinitialized.

3. The database log archiving method according to claim 1, characterized in that, After moving the target redo log file to a preset archive directory located on the same logical disk via MV operation, the process further includes: Change the name of the target redo log file according to the naming rules corresponding to the preset archive directory.

4. The database log archiving method according to claim 1, characterized in that, Creating a preliminary redo log file with the same attributes as the target redo log file and empty content includes: When a target redo log file that needs to be archived is identified, a file with the same attributes as the target redo log file and empty content is selected from the prepared file pool as a prepared redo log file. The prepared file pool is located on the same logical disk as the target redo log file, and contains multiple files with the same attributes as the target redo log file but empty content.

5. The database log archiving method according to claim 1, characterized in that, The step of creating a preliminary redo log file with the same attributes as the target redo log file and empty content on the same logical disk as the target redo log file also includes: Create a preliminary redo log file with the same attributes as the target redo log file and empty content, located on the same logical disk and in the same storage path as the target redo log file.

6. The database log archiving method according to claim 1, characterized in that, After creating a preliminary redo log file with the same attributes as the target redo log file and empty content, the process further includes: Establish a mapping relationship between the prepared redo log file and the target redo log; After changing the filename and storage path of the prepared redo log file to the original filename and storage path of the target redo log file through the MV operation, the method further includes: Based on the mapping relationship, verify whether the archived target redo log file and the prepared redo log file after changing the file name and storage path correspond; If the verification passes, the prepared redo log file with the changed file name and storage path will be mounted to the database; If the verification fails, the target redo log file that needs to be archived in the database will be re-identified.

7. The database log archiving method according to claim 6, characterized in that, After mounting the prepared redo log file (with modified filename and storage path) to the database, the process also includes: Clear the mapping relationship between the prepared redo log file and the target redo log.

8. A database log archiving device, characterized in that, include: The identification module is used to identify the target redo log files that the database needs to archive; A creation module is used to create a preliminary redo log file with the same attributes as the target redo log file and empty content, on the same logical disk as the target redo log file; The archiving module is used to move the target redo log file to a preset archive directory located on the same logical disk via MV operation; The module is used to change the filename and storage path of the prepared redo log file to the original filename and storage path of the target redo log file through MV operation; The mounting module is used to mount the prepared redo log file, after changing its file name and storage path, to the database.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the database log archiving method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the database log archiving method as described in any one of claims 1 to 7.

Images