Database backup method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a database backup method and device, electronic equipment and storage medium, the method comprising: in the case of reaching a preset time node, triggering the database to refresh the data page in the shared cache area to the disk, and locking the file operation of the database switching pre-write log; after creating the basic snapshot data of the database and storing the basic snapshot data, unlocking the file operation, and listening to a plurality of pre-write logs output by the database after the file operation recovers, generating incremental log data based on the plurality of pre-write logs and storing the incremental log data; determining the starting file identifier corresponding to the earliest pre-write log in the incremental log data, and sequentially marking other pre-write logs in the incremental log data based on the starting file identifier; creating metadata information corresponding to the basic snapshot data. The present scheme realizes the non-interrupted backup of the database, guarantees the data consistency and integrity, reduces the backup data volume, and improves the backup efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a database backup method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, an increasing number of network attached storage (NET) devices integrate object-relational databases (ORDs), which have become the core support for the transformation of NETD devices from simple file storage to intelligent storage. These databases are widely used in various scenarios, including home use, small and medium-sized enterprise offices, and small data centers. These integrated ORDs primarily store various types of critical structured data, including the NETD device's own application configurations (such as backup policies, sharing rules, and device operating parameters), user data indexes, multimedia information (such as photo metadata, video format parameters, and file category tags), and permission system records (such as user account passwords, shared folder access permissions, and operation audit logs). This data directly determines the normal operation and user experience of the NETD device. If the database is damaged, data is lost, or corrupted, the core functions of the NETD device, such as intelligent retrieval and permission management, will be paralyzed, and users will be unable to access and manage the stored data normally.

[0003] To mitigate potential risks such as hardware failures, software crashes, virus attacks, and misoperations faced by network-attached storage devices, and to prevent database crashes or data loss, related technologies periodically stop database services and perform backups by completely copying the database directory. This causes user access interruptions, affects business continuity, and has low backup efficiency, requiring improvement. Summary of the Invention

[0004] This application provides a database backup method, apparatus, electronic device, and storage medium, which solves the problems in related technologies where data block backup causes user access interruption, affects business continuity, and has low backup efficiency. It can achieve uninterrupted database backup, ensure data consistency and integrity, reduce the amount of backup data, and improve backup efficiency.

[0005] In a first aspect, embodiments of this application provide a database backup method, the method comprising:

[0006] Upon reaching a preset time point, the database is triggered to flush the data pages in the shared cache to the disk and lock the database's file operation for switching write-ahead logs;

[0007] After creating and storing the basic snapshot data of the database, the file operation is unlocked, and multiple write-ahead logs output by the database after the file operation is restored are monitored. Incremental log data is generated based on the multiple write-ahead logs and the incremental log data is stored.

[0008] Determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and mark the other write-ahead logs in the incremental log data in order based on the starting file identifier;

[0009] Metadata information corresponding to the basic snapshot data is created to complete the data backup of the database. The metadata information includes the starting file identifier.

[0010] Secondly, embodiments of this application also provide a database backup device, comprising:

[0011] The initial processing module is configured to trigger the database to flush the data pages in the shared cache to the disk and lock the database's file operation of switching write-ahead logs when a preset time node is reached.

[0012] The data generation module is configured to create and store basic snapshot data of the database, then unlock the file operation, listen to multiple write-ahead logs output by the database after the file operation is restored, generate incremental log data based on the multiple write-ahead logs, and store the incremental log data.

[0013] The data tagging module is configured to determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and to tag other write-ahead logs in the incremental log data in sequence based on the starting file identifier;

[0014] The data association module is configured to create metadata information corresponding to the basic snapshot data in order to complete the data backup of the database. The metadata information includes the starting file identifier.

[0015] Thirdly, embodiments of this application also provide an electronic device, the device comprising:

[0016] One or more processors;

[0017] Storage device, configured to store one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the database backup method described in the embodiments of this application.

[0019] Fourthly, embodiments of this application also provide a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are configured to perform the database backup method described in embodiments of this application.

[0020] In this embodiment, triggering the database to flush data pages in the shared cache to disk and locking the database switch write-ahead log file operation ensures the physical consistency of the database snapshot, achieving zero-downtime hot backup. Creating basic snapshot data for the database and generating corresponding incremental log data enables lightweight incremental backup, balancing the resource limitations of network-attached storage devices and backup real-time requirements. Determining the starting file identifier and sequentially marking other write-ahead logs in the incremental log data ensures the timeliness of the incremental logs, laying the foundation for subsequent point-in-time recovery. By creating metadata information corresponding to the basic snapshot data, the basic snapshot data and incremental log data can be associated to build a standardized backup chain, achieving automated management of backup data. The above solution enables uninterrupted database backup, ensures data consistency and integrity, reduces the amount of backup data, and improves backup efficiency. Attached Figure Description

[0021] Figure 1 A flowchart illustrating a database backup method provided in this application embodiment;

[0022] Figure 2 A flowchart of a database backup method including a data recovery process is provided for embodiments of this application;

[0023] Figure 3 A flowchart illustrating a specific implementation process for database recovery provided in this application embodiment;

[0024] Figure 4 A flowchart of a database backup method including another data recovery process provided for embodiments of this application;

[0025] Figure 5 A flowchart of a database backup method including a data verification process is provided for embodiments of this application;

[0026] Figure 6 A flowchart of a database backup method including an incremental data storage procedure is provided for embodiments of this application;

[0027] Figure 7 A flowchart illustrating a database backup method including an expired data filtering process, provided for embodiments of this application;

[0028] Figure 8 A structural block diagram of a database backup device provided in an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of this application and are not intended to limit the scope of the embodiments. Furthermore, it should be noted that, for ease of description, only the parts relevant to the embodiments of this application are shown in the accompanying drawings, not the entire structure.

[0031] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0032] The database backup method, apparatus, electronic device, and storage medium provided in this application are applicable to embedded object-relational database systems running in scenarios such as home network attached storage, enterprise private cloud network attached storage, and edge computing devices, such as PostgreSQL, but are not limited to these in this application.

[0033] Figure 1 A flowchart of a database backup method provided in this application embodiment is shown below. Figure 1 As shown, this database backup method specifically includes the following steps:

[0034] Step S101: When the preset time node is reached, the database is triggered to flush the data pages in the shared cache to the disk and lock the database to switch the file operation of the write-ahead log.

[0035] The preset time node can be a pre-configured time point to trigger the backup process, specifically configured as a fixed time, periodic time, or trigger-based time condition, etc., which are not limited in this application. Since the database defaults to asynchronous disk flushing, the memory data is updated relative to the disk data. However, the base snapshot data for subsequent steps can only reference the disk data. Therefore, it is necessary to actively trigger disk flushing to ensure that the disk data is consistent with the memory data at the start of the backup. Thus, if the preset time node is reached, a checkpoint command can be sent to the database to execute the relevant operations, specifically triggering the database to flush data pages in the shared cache to disk. The shared cache can be a cache area set up in memory for temporarily storing frequently accessed data pages, and a data page can be the smallest physical storage unit for the database to manage disk data. By flushing the data pages in the shared cache to disk, it can be ensured that the disk data is consistent with the memory data at the start of the backup, avoiding data loss and providing a complete data foundation for the creation of subsequent base snapshot data. Furthermore, since the data capacity of a single write-ahead log is fixed, if the written data reaches a preset threshold, it will be considered full. During the creation of basic snapshot data, the database may generate new write-ahead logs due to the write-ahead log being full. This can prevent the basic snapshot data from clearly identifying its corresponding starting write-ahead log, thus affecting the determination of incremental logs. Therefore, in addition to triggering the database to flush data pages in the shared cache to disk, it is also necessary to lock the database's file operations for switching write-ahead logs. Specifically, this can be achieved by embedding a self-developed hook function or a custom extension plugin at the kernel level, temporarily freezing the database's file operations for switching write-ahead logs, while still allowing business-related CRUD operations to be written normally to the current write-ahead log. This avoids write-ahead log switching during subsequent basic snapshot data creation, prevents the loss or gap of the incremental log's starting file identifier, and ensures that the lock duration is extremely short, only covering the window period before snapshot creation, without affecting business log recording and minimizing the impact on database performance.

[0036] Step S102: After creating and storing the basic snapshot data of the database, unlock file operations, and listen to multiple write-ahead logs output by the database after the file operations are restored. Generate incremental log data based on the multiple write-ahead logs and store the incremental log data.

[0037] The basic snapshot data can be a complete data image at the point in time when the write-ahead log (WAL) is locked, containing all core content of the database, such as all data files, configuration files, system tables, and data pages. It serves as the data baseline for subsequent incremental backups. Specifically, creating the basic snapshot data can be done by calling the snapshot API of the underlying file system (e.g., ext4, btrfs, or ZFS) to generate a database snapshot. This is an instantaneous operation, taking milliseconds, without interrupting database services. The basic snapshot data can be stored on local storage media or a dedicated backup partition. After the basic snapshot data is created and stored, the temporary restrictions on switching the write-ahead log can be lifted, restoring the database to normal operation and achieving backup without downtime. Based on the point in time after file operation recovery, multiple write-ahead logs output by the database can be continuously monitored. Incremental log data can be structured incremental backup data that can be directly used for recovery, formed by lightweight processing of the raw write-ahead logs collected through monitoring. This processing can include lightweight compression, deduplication detection, and breakpoint resume marking. Incremental log data can be stored in a separate directory from the basic snapshot data, or it can be synchronized to an off-site backup node. The storage process supports breakpoint resumption; if the network-attached storage device experiences a network outage or power failure, storage can resume after recovery without data loss. Therefore, by recording only data modifications made after the snapshot, the storage and computing power consumption of the network-attached storage device is significantly reduced. Simultaneously, standardized incremental log data is generated, providing a directly usable data source for subsequent point-in-time recovery and off-site disaster recovery.

[0038] Step S103: Determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and mark the other write-ahead logs in the incremental log data in sequence based on the starting file identifier.

[0039] The earliest write-ahead log (WAL) can be the first WAL generated by the database after the creation of the basic snapshot data and the unlocking of the file, serving as the starting point for incremental log data. The starting file identifier can be a globally unique identifier assigned to the WAL, serving as its unique name. Sequential marking involves adding an offset number to all subsequent WALs in the incremental log, based on the starting file identifier, creating a continuous and ordered log chain. For example, if the starting file identifier is 00000001000000000000003A, and other WALs are subsequently generated during database operation, these WALs can be archived sequentially as 0000000100000000000003B, 00000001000000000000003C, 0000000100000000000003D, etc., serving as a key basis for distinguishing different WALs and determining log order. This establishes a strict temporal relationship for incremental log data, forming a structured and orderly incremental log chain.

[0040] Step S104: Create metadata information corresponding to the basic snapshot data to complete the database data backup. The metadata information includes the starting file identifier.

[0041] The metadata information can be descriptive data about the basic snapshot data. This metadata information may include a start file identifier, used to associate the basic snapshot data with its subsequent incremental log data. It can be understood that the start file identifier is the temporal starting point of the incremental write-ahead log. With this start file identifier included in the metadata, the system can quickly locate all write-ahead log files corresponding to the basic snapshot data. Optionally, the metadata information may also include a snapshot identifier, a timestamp identifier, and a recoverable status. Thus, the basic snapshot data and incremental log data form a complete data backup chain through the start file identifier.

[0042] The above-described steps—triggering the database to flush data pages from the shared cache to disk and locking the database's write-ahead log file operations—ensure the physical consistency of the database snapshot, achieving zero-downtime hot backups. Creating basic snapshot data and generating corresponding incremental log data enables lightweight incremental backups, balancing resource limitations of network-attached storage devices with backup real-time requirements. Determining the starting file identifier and sequentially marking other write-ahead logs in the incremental log data ensures the timeliness of the incremental logs, laying the foundation for subsequent point-in-time recovery. By creating metadata information corresponding to the basic snapshot data, the basic snapshot data and incremental log data can be associated, constructing a standardized backup chain and achieving automated backup data management. This solution enables uninterrupted database backups, ensuring data consistency and integrity, reducing the amount of backup data, and improving backup efficiency.

[0043] Figure 2 A flowchart of a database backup method including a data recovery process is provided for embodiments of this application, as shown below. Figure 2 As shown, this database backup method specifically includes the following steps:

[0044] Step S201: When the preset time node is reached, the database is triggered to flush the data pages in the shared cache to the disk and lock the database to switch the file operation of the write-ahead log.

[0045] Step S202: After creating and storing the basic snapshot data of the database, unlock file operations, and listen to multiple write-ahead logs output by the database after the file operations are restored. Generate incremental log data based on the multiple write-ahead logs and store the incremental log data.

[0046] Step S203: Determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and mark the other write-ahead logs in the incremental log data in sequence based on the starting file identifier.

[0047] Step S204: Create metadata information corresponding to the basic snapshot data to complete the database data backup. The metadata information includes the starting file identifier.

[0048] Step S205: If the first recovery operation of the database is triggered, stop running the target database instance corresponding to the database.

[0049] The first recovery operation can be a database data restoration operation initiated by the user to the backup system, specifically a point-in-time recovery based on basic snapshot data and incremental log data. The target database instance can be a running database instance on a network-attached storage device that requires data recovery. If the target database instance is still running during the recovery process, it will continue to read and write data files. At this time, the snapshot overwriting and log replay of the recovery operation will conflict with normal read and write operations, leading to data file corruption or even recovery failure. Therefore, it is necessary to stop the running target database instance, specifically by terminating the processes and related services of the target database instance to provide a conflict-free execution environment for the recovery operation.

[0050] Step S206: Obtain the first basic snapshot data selected by the first recovery operation, and restore the data corresponding to the target database instance to the recovery time point specified by the first recovery operation based on the first basic snapshot data and the associated first incremental log data.

[0051] The first base snapshot data can be the base snapshot data selected by the user or system in the first recovery operation as the recovery baseline. Different base snapshots at different time points can be selected according to recovery needs. The first incremental log data can be the incremental log data associated with the first base snapshot data through a start file identifier; it is the core incremental data for recovery to the specified time point. The recovery time point can be the specific time node specified in the first recovery operation to which the database needs to be restored. Obtaining the first base snapshot data and associated first incremental log data selected in the first recovery operation can be data read from local or off-site storage locations. The specific restoration can be an operation that restores the database data to the specified time point through snapshot overwriting and log playback.

[0052] In one optional embodiment, a specific implementation process for database recovery is described; please refer to [reference needed]. Figure 3 This is a flowchart illustrating a specific implementation process for restoring a database, as provided in an embodiment of this application. Figure 3 As shown, the specific implementation steps for restoring the data corresponding to the target database instance to the recovery time point specified in the first recovery operation, based on the first basic snapshot data and the associated first incremental log data, include:

[0053] Step S2061: Determine the target data directory corresponding to the target database instance.

[0054] The target data directory can be the core physical storage directory of the target database instance running on the network-attached storage device. It is the dedicated directory where the database stores all core data files, configuration files, write-ahead log files, tablespace files, and system table files, and it is also the basic directory for the database instance to start and run. In one embodiment, the absolute path of the target data directory can be determined by reading the configuration file of the target database instance and parsing the parameter values ​​of the configuration items. In another embodiment, the pre-stored basic configuration information of the target database instance can be retrieved from the backup management system's backend configuration library of the network-attached storage device, and the path of its bound target data directory can be directly extracted.

[0055] Step S2062: Restore the data files in the first basic snapshot data to the target data directory, and determine the associated first incremental log data based on the metadata information corresponding to the first basic snapshot data.

[0056] The data files can be the core physical storage files of the database, representing the core content stored in the first basic snapshot. These include heap data files, index files, system table files, tablespace files, database configuration files, etc., forming the physical foundation for the database instance's operation. They are stored in binary format in the target data directory. Restoring the data files from the first basic snapshot to the target data directory involves decompressing and verifying the compressed or encapsulated physical data files from the first basic snapshot, then copying them completely to the target data directory, overwriting any existing damaged or missing files. Since the first basic snapshot data and the first incremental log data are associated through metadata information, the associated first incremental log data can be determined based on the metadata information corresponding to the first basic snapshot data. Specifically, by parsing the metadata information of the first basic snapshot, the incremental log association identifier and storage path can be extracted, allowing for precise matching of the corresponding first incremental log data.

[0057] Step S2063: Re-execute the write operations recorded in the first incremental log data according to the generation order of the first incremental log data until the recovery time point specified by the first recovery operation is reached.

[0058] The generation order can be the timestamp order or the ascending order of log sequence numbers generated by the database during business execution, perfectly matching the actual business operation order of the database. Write operations can be all data modification operations recorded in binary or structured form in the first incremental log data, including operations that change database data such as inserting, updating, deleting, and creating objects. The recovery time point can be the precise time node specified in the first recovery operation to which the database needs to be restored. This time point must be between the creation time of the first base snapshot and the latest generation time of the incremental log, and is the termination node of the write-ahead log replay. By reproducing and executing the write operations recorded in the incremental log sequentially on the restored base snapshot data, it is equivalent to reapplying the data modification instructions in the log to the physical data files in the target data directory.

[0059] This enables precise recovery based on the physical data directory, ensures the native compatibility of the database instance, and re-executes write operations according to the incremental log generation order to guarantee strong data consistency after recovery.

[0060] Step S207: Rerun the target database instance.

[0061] After the database data is restored, the processes and related services of the target database instance are started to restore normal database read and write services.

[0062] Based on the first basic snapshot data and the associated first incremental log data, the normal business services of the database can be quickly restored, significantly reducing the NAS business interruption time caused by data failure and ensuring that the database can provide normal read and write services.

[0063] Figure 4 A flowchart of a database backup method including another data recovery process is provided for embodiments of this application, such as... Figure 4 As shown, this database backup method specifically includes the following steps:

[0064] Step S401: When the preset time node is reached, the database is triggered to flush the data pages in the shared cache to the disk and lock the database to switch the file operation of the write-ahead log.

[0065] Step S402: After creating and storing the basic snapshot data of the database, unlock file operations, and listen to multiple write-ahead logs output by the database after the file operations are restored. Generate incremental log data based on the multiple write-ahead logs and store the incremental log data.

[0066] Step S403: Determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and mark the other write-ahead logs in the incremental log data in sequence based on the starting file identifier.

[0067] Step S404: Create metadata information corresponding to the basic snapshot data to complete the database data backup. The metadata information includes the starting file identifier.

[0068] Step S405: In the event of triggering the second recovery operation of the database, stop running the target database instance corresponding to the database and determine the target data directory corresponding to the target database instance.

[0069] The second recovery operation can be initiated by the user or the network-attached storage system. It restores the database to the snapshot creation time based solely on a consistent snapshot, without incremental log parsing and replay steps. This operation is triggered when complete data recovery is required. Accordingly, the target database instance in this embodiment can be a running database instance in the network-attached storage device that requires complete data recovery. If the target database instance is still running during the recovery process, it will continue to read and write data files. At this time, the snapshot overwrite of the recovery operation conflicts with normal read and write operations, leading to data file corruption or even recovery failure. Therefore, it is necessary to stop the running target database instance, specifically by terminating the process and related services of the target database instance to provide a conflict-free execution environment for the recovery operation.

[0070] Step S406: Obtain the second basic snapshot data selected in the second recovery operation, and restore the data files in the second basic snapshot data to the target data directory.

[0071] The second base snapshot data can be the base snapshot data selected by the user or system during the second recovery operation, serving as the recovery benchmark. Different time points can be selected as the base snapshots based on recovery needs. Data files can be the core physical storage files of the database, representing the core content stored in the second base snapshot. These include heap data files, index files, system table files, tablespace files, database configuration files, etc., forming the physical basis for the database instance's operation, and are stored in binary format in the target data directory. Restoring the data files from the second base snapshot data to the target data directory can specifically involve decompressing and verifying the compressed or encapsulated physical data files from the second base snapshot, then completely copying them to the target data directory, overwriting any existing damaged or missing files.

[0072] Step S407: Rerun the target database instance.

[0073] After the database data is restored, the processes and related services of the target database instance are started to restore normal database read and write services.

[0074] The above approach can meet the data processing needs for complete recovery, directly restoring data based on the second basic snapshot data and quickly restoring the data state to the snapshot time point.

[0075] Figure 5 A flowchart of a database backup method including a data verification process is provided for embodiments of this application, as shown below. Figure 5 As shown, this database backup method specifically includes the following steps:

[0076] Step S501: When the preset time node is reached, the database is triggered to flush the data pages in the shared cache to the disk and lock the database to switch the file operation of the write-ahead log.

[0077] Step S502: After creating and storing the basic snapshot data of the database, unlock file operations, and listen to multiple write-ahead logs output by the database after the file operations are restored. Generate incremental log data based on the multiple write-ahead logs and store the incremental log data.

[0078] Step S503: Determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and mark the other write-ahead logs in the incremental log data in sequence based on the starting file identifier.

[0079] Step S504: Create metadata information corresponding to the basic snapshot data to complete the database data backup. The metadata information includes the starting file identifier.

[0080] Step S505: Obtain the third basic snapshot data and the third incremental log data generated at the target time node.

[0081] The target time node can be the time node selected based on the actual verification requirements for the backup data generation. This target time node is the creation time of the third basic snapshot data, and the third incremental log data is associated with this third basic snapshot data.

[0082] Step S506: Verify whether the target structure and metadata of the third basic snapshot data are complete, and verify whether the third incremental log data is continuous and parseable.

[0083] The target structure can be the native data file directory structure that the third-level basic snapshot data should possess, including core directory levels such as the database heap data file directory, index file directory, system table directory, and configuration file directory. This is the physical basis for the recoverability and rollback of the snapshot data. Meta-information can be structured descriptive data bound to the third-level basic snapshot data, including key fields such as snapshot creation time, storage path, and the starting file identifier of the associated incremental log. This is the core basis for the association between snapshots and logs and for verifying data integrity. Specifically, verifying the completeness of the target structure and meta-information of the third-level basic snapshot data can involve checking whether its directory structure is consistent with the native data directory structure, checking for missing core data files and configuration files, and parsing the meta-information file to verify whether key fields (such as the starting file identifier) ​​are complete and free of null values. Verifying the continuity and parsability of the third-level incremental log data can specifically involve verifying whether each write-ahead log in the third-level incremental log data follows an increasing order of log sequence number or generation timestamp without any breaks or omissions, and whether the write, modify, and delete operation commands recorded in the logs can be parsed and executed normally, without file corruption, format errors, or data garbled characters.

[0084] Step S507: If the target structure and metadata are complete, and the third incremental log data is continuous and parseable, start the target database instance corresponding to the database, and roll back the target database instance to the specified time node based on the third basic snapshot data and the third incremental log data.

[0085] If the target structure and metadata are complete, and the third incremental log data is continuous and parseable, the third basic snapshot data and the third incremental log data can be considered to have passed static verification. Furthermore, after the backup data verification passes, the process and background persistent services of the target database instance are started to provide the runtime environment for subsequent rollback operations. The specified time node can be a database rollback target time point selected by the user based on actual rollback needs, later than the target time node (snapshot creation time). Rolling back the target database instance to the specified time node can specifically be done by restoring snapshots and sequentially replaying incremental logs to the specified time point, restoring it to the normal data state of the specified time node.

[0086] Step S508: If the rollback of the target database instance is successful, mark the third basic snapshot data and the third incremental log data as recoverable.

[0087] If the target database instance rollback is successful, the third basic snapshot data and the third incremental log data can be considered to have passed dynamic verification. Combined with step S507, it can be confirmed that the third basic snapshot data and the third incremental log data are valid backup data, meeting the requirements for database recovery and rollback. Specifically, marking the third basic snapshot data and the third incremental log data as recoverable can be achieved by adding status tags to the verified valid third basic snapshot data and the third incremental log data to indicate data validity.

[0088] The above-mentioned verification of the third basic snapshot data and the third incremental log data can help retain valid backup data and improve the overall reliability of the backup chain.

[0089] Figure 6 A flowchart of a database backup method including an incremental data storage procedure is provided for embodiments of this application, as shown below. Figure 6 As shown, this database backup method specifically includes the following steps:

[0090] Step S601: When the preset time node is reached, the database is triggered to flush the data pages in the shared cache to the disk and lock the database to switch the file operation of the write-ahead log.

[0091] Step S602: After creating and storing the basic snapshot data of the database, unlock file operations, and listen to multiple write-ahead logs output by the database after the file operations are restored. Generate incremental log data based on the multiple write-ahead logs and store the incremental log data.

[0092] Step S603: Determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and mark the other write-ahead logs in the incremental log data in sequence based on the starting file identifier.

[0093] Step S604: Create metadata information corresponding to the basic snapshot data to complete the database data backup. The metadata information includes the starting file identifier.

[0094] Step S605: Perform block verification and comparison between the basic snapshot data and incremental log data and the historical backup data of the external storage device to obtain incremental data blocks.

[0095] Historical backup data can be stored on external storage devices and belong to the same backup chain as the current basic snapshot data and incremental log data. The block verification comparison involves splitting the basic snapshot data and incremental log data into fixed-size independent data blocks, calculating a unique checksum for each block using algorithms such as MD5 and SHA256, and comparing the checksums of the split data blocks with those of the corresponding historical backup data blocks to determine consistency. If the checksum of a split data block differs from that of the corresponding historical backup data block, that split data block can be identified as an incremental data block. This incremental data block can be a filtered difference or a newly added data block.

[0096] Step S606: Transfer the incremental data block to the external storage device so that the external storage device can reconstruct the data based on the incremental data block to obtain the basic snapshot data and incremental log data.

[0097] After the incremental data blocks are transferred to the external storage device, the external storage device can use historical backup data blocks as a basis to replace the difference blocks or add new blocks to restore complete and consistent basic snapshot data and incremental log data. In one embodiment, the position of the corresponding block can be located in the historical backup data, and the difference block can be replaced with the incremental data block. In another embodiment, the incremental data block can be added to the corresponding position in the historical backup data.

[0098] As described above, by transmitting only incremental data blocks, the amount of data transmitted can be significantly reduced, transmission efficiency can be improved, disaster recovery data consistency can be ensured, and recovery reliability can be guaranteed.

[0099] Figure 7 A flowchart of a database backup method including an expired data filtering process is provided for embodiments of this application, as shown below. Figure 7 As shown, this database backup method specifically includes the following steps:

[0100] Step S701: When the preset time node is reached, the database is triggered to flush the data pages in the shared cache to the disk and lock the database to switch the file operation of the write-ahead log.

[0101] Step S702: After creating and storing the basic snapshot data of the database, unlock file operations, and listen to multiple write-ahead logs output by the database after the file operations are restored. Generate incremental log data based on the multiple write-ahead logs and store the incremental log data.

[0102] Step S703: Determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and mark the other write-ahead logs in the incremental log data in sequence based on the starting file identifier.

[0103] Step S704: Create metadata information corresponding to the basic snapshot data to complete the database data backup. The metadata information includes the starting file identifier.

[0104] Step S705: Select target basic snapshot data from multiple stored basic snapshot data that has been stored for a longer than the preset retention time.

[0105] The stored base snapshot data can be a set of full, consistent base snapshots generated at a preset period. The preset retention time can be a backup data retention threshold configured by the user or system to balance data retention duration and storage resource usage. The target base snapshot data can be expired base snapshots that have been stored for longer than the preset retention time. By filtering the target base snapshot data, the latest backup can be retained first.

[0106] Step S706: Determine the associated target incremental log data based on the metadata information corresponding to the target basic snapshot data.

[0107] Metadata information serves as the basis for establishing a unique association between basic snapshot data and incremental log data. For example, metadata information includes a starting file identifier, which can locate the associated target incremental log data.

[0108] Step S707: Delete the target basic snapshot data and the target incremental log data.

[0109] As described above, by removing the target basic snapshot data and the target incremental log data, expired data can be cleared in a timely manner, alleviating storage capacity pressure, ensuring the timeliness of backup data, and balancing data retention time with NAS storage resource usage.

[0110] Figure 8 This is a structural block diagram of a database backup device provided in an embodiment of this application. The device is configured to execute the database backup method provided in the above embodiment, and has corresponding functional modules and beneficial effects for executing the method. For example... Figure 8 As shown, the device specifically includes:

[0111] The initial processing module 801 is configured to trigger the database to flush the data pages in the shared cache to the disk and lock the database to switch the file operation of the write-ahead log when a preset time node is reached;

[0112] The data generation module 802 is configured to create and store basic snapshot data of the database, then unlock file operations, listen to multiple write-ahead logs output by the database after file operations are restored, generate incremental log data based on the multiple write-ahead logs, and store the incremental log data.

[0113] The data tagging module 803 is configured to determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and to tag other write-ahead logs in the incremental log data in sequence based on the starting file identifier;

[0114] The data association module 804 is configured to create metadata information corresponding to the basic snapshot data in order to complete the database data backup. The metadata information includes the starting file identifier.

[0115] The above-described steps—triggering the database to flush data pages from the shared cache to disk and locking the database's write-ahead log file operations—ensure the physical consistency of the database snapshot, achieving zero-downtime hot backups. Creating basic snapshot data and generating corresponding incremental log data enables lightweight incremental backups, balancing resource limitations of network-attached storage devices with backup real-time requirements. Determining the starting file identifier and sequentially marking other write-ahead logs in the incremental log data ensures the timeliness of the incremental logs, laying the foundation for subsequent point-in-time recovery. By creating metadata information corresponding to the basic snapshot data, the basic snapshot data and incremental log data can be associated, constructing a standardized backup chain and achieving automated backup data management. This solution enables uninterrupted database backups, ensuring data consistency and integrity, reducing the amount of backup data, and improving backup efficiency.

[0116] In one possible embodiment, a first recovery module is also included, configured as follows:

[0117] If the first database recovery operation is triggered, the target database instance corresponding to the database will be stopped.

[0118] Obtain the first basic snapshot data selected in the first recovery operation, and restore the data corresponding to the target database instance to the recovery time point specified in the first recovery operation based on the first basic snapshot data and the associated first incremental log data;

[0119] Rerun the target database instance.

[0120] In one possible embodiment, the first recovery module is specifically configured as follows:

[0121] Determine the target data directory corresponding to the target database instance;

[0122] The data files in the first basic snapshot data are restored to the target data directory, and the associated first incremental log data is determined based on the metadata information corresponding to the first basic snapshot data.

[0123] Following the generation order of the first incremental log data, re-execute the write operations recorded in the first incremental log data until the recovery time point specified by the first recovery operation is reached.

[0124] In one possible embodiment, a second recovery module is also included, configured as follows:

[0125] In the event of triggering the second database recovery operation, stop running the target database instance corresponding to the database and determine the target data directory corresponding to the target database instance;

[0126] Obtain the second base snapshot data selected in the second recovery operation, and restore the data files in the second base snapshot data to the target data directory;

[0127] Rerun the target database instance.

[0128] In one possible embodiment, a data verification module is also included, configured as follows:

[0129] Obtain the third basic snapshot data and the third incremental log data generated at the target time point;

[0130] Verify the completeness of the target structure and metadata of the third basic snapshot data, and verify the continuity and parsability of the third incremental log data;

[0131] If the target structure and metadata are complete, and the third incremental log data is continuous and parseable, start the target database instance corresponding to the database, and roll back the target database instance to the specified time node based on the third basic snapshot data and the third incremental log data.

[0132] If the rollback to the target database instance is successful, mark the third basic snapshot data and the third incremental log data as recoverable.

[0133] In one possible embodiment, an incremental transmission module is also included, configured as follows:

[0134] The incremental data blocks are obtained by comparing the basic snapshot data and incremental log data with the historical backup data of the external storage device in blocks.

[0135] Incremental data blocks are transferred to an external storage device so that the external storage device can reconstruct the data based on the incremental data blocks to obtain basic snapshot data and incremental log data.

[0136] In one possible embodiment, a data filtering module is also included, configured to:

[0137] Filter out target basic snapshot data from multiple stored basic snapshot data that has been stored for a longer than a preset retention period;

[0138] Determine the associated target incremental log data based on the metadata information corresponding to the target base snapshot data;

[0139] Delete the target base snapshot data and the target incremental log data.

[0140] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, the device includes a processor 901, a memory 902, an input device 903, and an output device 904; the number of processors 901 in the device can be one or more. Figure 9 Taking a processor 901 as an example; the processor 901, memory 902, input device 903, and output device 904 in the device can be connected via a bus or other means. Figure 9 Taking a bus connection as an example, the memory 902, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the database backup method in this embodiment. The processor 901 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 902, thereby implementing the aforementioned database backup method. The input device 903 can be configured to receive input digital or character information and generate key signal inputs related to user settings and function control of the device. The output device 904 may include a display screen or other display device.

[0141] The electronic device provided above can be used to execute the database backup method provided in any of the above embodiments, and has the corresponding functions and beneficial effects.

[0142] This application also provides a non-volatile storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are configured to perform a database backup method described in the above embodiments. The method includes: upon reaching a preset time node, triggering the database to flush data pages in the shared cache to disk and locking file operations that switch the database write-ahead log; creating and storing basic snapshot data of the database, then unlocking the file operations and monitoring multiple write-ahead logs output by the database after file operations are restored; generating and storing incremental log data based on the multiple write-ahead logs; determining the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and sequentially marking other write-ahead logs in the incremental log data based on the starting file identifier; and creating metadata information corresponding to the basic snapshot data to complete the database backup. The metadata information includes the starting file identifier.

[0143] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media, or optical storage; registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term “storage medium” may include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0144] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the database backup method described above, but can also perform related operations in the database backup method provided in any embodiment of this application.

[0145] It is worth noting that in the above-described embodiments of the database backup device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not configured to limit the protection scope of the embodiments of this application.

[0146] It should be noted that the numbering of each step in this solution is only used to describe the overall design framework of this solution and does not indicate a necessary sequential relationship between the steps. As long as the overall implementation process conforms to the overall design framework of this solution, it falls within the protection scope of this solution. The order of the text in the description is not an exclusive limitation on the specific implementation process of this solution. Those skilled in the art should understand that the embodiments of this application can be provided as methods, systems, or computer program products. In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent memory in computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0147] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0148] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A database backup method, characterized in that, The method includes: When a preset time node is reached, the database is triggered to flush the data pages in the shared cache to the disk and lock the file operation of the database switch write-ahead log. The shared cache is a cache area set up in memory for temporary storage of frequently accessed data pages. Locking the file operation of the database switch write-ahead log is achieved by implanting a self-developed hook function or a custom extension plugin in the kernel layer, so that the file operation of the database switch write-ahead log is temporarily frozen, but the CRUD operations generated by the business can still be written to the current write-ahead log normally. After creating and storing the basic snapshot data of the database, the file operation is unlocked, and multiple write-ahead logs output by the database after the file operation is restored are monitored. Incremental log data is generated based on the multiple write-ahead logs and the incremental log data is stored. The basic snapshot data is a complete data image at the time of locking the write-ahead log switch, which includes all data files, configuration files, system tables and data pages of the database. The starting file identifier corresponding to the earliest write-ahead log in the incremental log data is determined, and other write-ahead logs in the incremental log data are marked in sequence based on the starting file identifier. The incremental log data is structured incremental backup data that can be directly used for recovery after lightweight processing of the original write-ahead logs collected by monitoring. The processing includes lightweight compression, duplicate detection and breakpoint resume marking. Create metadata information corresponding to the basic snapshot data to complete the data backup of the database. The metadata information includes the starting file identifier. After completing the data backup of the database, the process also includes: If the first recovery operation of the database is triggered, the target database instance corresponding to the database will be stopped. Obtain the first basic snapshot data selected by the first recovery operation, and based on the first basic snapshot data and the associated first incremental log data, restore the data corresponding to the target database instance to the recovery time point specified by the first recovery operation; Rerun the target database instance; The step of restoring the data corresponding to the target database instance to the recovery time point specified in the first recovery operation based on the first basic snapshot data and the associated first incremental log data includes: Determine the target data directory corresponding to the target database instance; The data files in the first basic snapshot data are restored to the target data directory, and the associated first incremental log data is determined based on the metadata information corresponding to the first basic snapshot data. Following the generation order of the first incremental log data, re-execute the write operations recorded in the first incremental log data until the recovery time point specified by the first recovery operation is reached.

2. The database backup method according to claim 1, characterized in that, After completing the data backup of the database, the following is also included: If the second recovery operation of the database is triggered, stop running the target database instance corresponding to the database and determine the target data directory corresponding to the target database instance; Obtain the second base snapshot data selected in the second recovery operation, and restore the data files in the second base snapshot data to the target data directory; Rerun the target database instance.

3. The database backup method according to claim 1, characterized in that, After completing the data backup of the database, the following is also included: Obtain the third basic snapshot data and the third incremental log data generated at the target time point; Verify whether the target structure and metadata of the third basic snapshot data are complete, and verify whether the third incremental log data is continuous and parseable; If the target structure and the metadata are complete, and the third incremental log data is continuous and parseable, start the target database instance corresponding to the database, and roll back the target database instance to the specified time node based on the third basic snapshot data and the third incremental log data; If the rollback of the target database instance is successful, the third basic snapshot data and the third incremental log data are marked as recoverable.

4. The database backup method according to claim 1, characterized in that, After completing the data backup of the database, the following is also included: The basic snapshot data and the incremental log data are respectively compared with the historical backup data of the external storage device to obtain incremental data blocks. The incremental data block is transmitted to the external storage device so that the external storage device can reconstruct the data based on the incremental data block to obtain the basic snapshot data and the incremental log data.

5. The database backup method according to claim 1, characterized in that, After completing the data backup of the database, the following is also included: Filter out target basic snapshot data from multiple stored basic snapshot data that has been stored for a longer than a preset retention period; The associated target incremental log data is determined based on the metadata information corresponding to the target basic snapshot data. Delete the target base snapshot data and the target incremental log data.

6. A database backup device, characterized in that, include: The initial processing module is configured to trigger the database to flush the data pages in the shared cache to the disk when a preset time node is reached, and to lock the file operation of the database switching write-ahead log. The shared cache is a cache area set in memory for temporarily storing frequently accessed data pages. Locking the file operation of the database switching write-ahead log is achieved by implanting a self-developed hook function or a custom extension plugin in the kernel layer, so that the file operation of the database switching write-ahead log is temporarily frozen, but the CRUD operations generated by the business can still be written to the current write-ahead log normally. The data generation module is configured to create and store basic snapshot data of the database, then unlock the file operation, listen to multiple write-ahead logs output by the database after the file operation is restored, generate incremental log data based on the multiple write-ahead logs, and store the incremental log data. The basic snapshot data is a complete data image at the time of locking the write-ahead logs, including all data files, configuration files, system tables, and data pages of the database. The data tagging module is configured to determine the starting file identifier corresponding to the earliest write-ahead log in the incremental log data, and to tag other write-ahead logs in the incremental log data in sequence based on the starting file identifier. The incremental log data is structured incremental backup data that can be directly used for recovery after lightweight processing of the original write-ahead logs collected by monitoring. The processing includes lightweight compression, duplicate detection and breakpoint resume tagging. The data association module is configured to create metadata information corresponding to the basic snapshot data in order to complete the data backup of the database. The metadata information includes the starting file identifier. The first recovery module is configured to, upon triggering the first recovery operation of the database, stop the running of the target database instance corresponding to the database; obtain the first basic snapshot data selected by the first recovery operation; restore the data corresponding to the target database instance to the recovery time point specified by the first recovery operation based on the first basic snapshot data and the associated first incremental log data; and rerun the target database instance. The first recovery module is specifically configured to: determine the target data directory corresponding to the target database instance; restore the data files in the first basic snapshot data to the target data directory; determine the associated first incremental log data based on the metadata information corresponding to the first basic snapshot data; and re-execute the write operations recorded in the first incremental log data according to the generation order of the first incremental log data until the recovery time point specified by the first recovery operation is reached.

7. An electronic device, characterized in that, The electronic device includes: one or more processors; and a memory configured to store one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the database backup method according to any one of claims 1-5.

8. A non-volatile storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are configured to perform the database backup method according to any one of claims 1-5.

Images