A method, device, equipment and medium for database streaming backup breakpoint continuation

Abstract

The application discloses a database streaming backup breakpoint continuation method and device, equipment and medium, and relates to the technical field of database operation and maintenance, which comprises the following steps: taking a database snapshot by starting a preset physical backup tool, and saving a list of data files to be backed up in the current batch to a preset list file in the local server based on the database snapshot information; performing file compression, transmission and state labeling based on the preset list file and the token bucket algorithm until the files are successfully uploaded to the backup server; if the preset physical backup tool is started again within a preset time period, it is judged whether the preset list file of the last batch and the database snapshot information exist in the local server; when the judgment result indicates that they exist, a backup type selection prompt is given, and if the user selection result received is breakpoint continuation, a breakpoint continuation process is triggered based on the preset list file of the last batch, the corresponding data file state labeling information and the database snapshot information. The database backup efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of database operation and maintenance technology, and in particular to a method, apparatus, device and medium for database streaming backup with breakpoint resume. Background Technology

[0002] Currently, a common method for database disaster recovery is to regularly back up the data stored in the database. When a database crashes and cannot be read correctly, the backup data can be restored to the database, thereby quickly restoring database business functions and effectively ensuring the security and integrity of user business data stored in the database, preventing large-scale data loss.

[0003] However, due to the massive amount of business data accumulated in the database daily, database backups can be time-consuming, often taking hours, and can put significant pressure on network transmission during the process. Furthermore, the long backup process and high network transmission pressure can lead to network interruptions during backup. Conventional backup methods would then require re-initiating the backup command and restarting the process, re-transferring the backup data from the previous interruption. This creates unnecessary network transmission pressure and generates junk backup data on the server hosting the backup data. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for database streaming backup with breakpoint resumption, which can effectively avoid repeated transmission of backup data, improve database backup efficiency, and reduce the impact on database server storage space. The specific solution is as follows:

[0005] Firstly, this application provides a method for resuming interrupted data transfers during database streaming backup, which should include:

[0006] By launching the preset physical backup tool, a database snapshot is taken, and based on the obtained database snapshot information, the list of data files to be backed up in this batch is saved to the corresponding preset list file on the local server.

[0007] Based on the preset list file, token bucket algorithm, and preset file physical backup rules, file compression, file transfer, and file status labeling are performed until all data files in this batch are successfully uploaded to the corresponding backup server, completing the corresponding streaming backup operation and obtaining the corresponding data file status labeling information.

[0008] If the preset physical backup tool is restarted again within a preset time period, it is determined whether the preset list file and the database snapshot information of the previous batch exist on the local server, so as to obtain the corresponding judgment result.

[0009] When the judgment result indicates that a backup type selection prompt is given, if the received user selection result is "resumable from breakpoint", the corresponding resume from breakpoint process is triggered based on the preset list files of the previous batch, the corresponding data file status label information, and the database snapshot information to complete the corresponding file backup.

[0010] Optionally, the step of launching a preset physical backup tool to take a database snapshot, and saving the list of data files to be backed up in this batch to a corresponding preset list file on the local server based on the obtained database snapshot information, includes:

[0011] The preset physical backup tool is launched based on the corresponding tool launch parameters; wherein, the tool launch parameters include the corresponding database administrator account information and input parameters used to characterize the batch backup file size threshold;

[0012] The preset physical backup tool captures the complete state of the database data files, log files, and control files at any given moment, performs a database snapshot, and obtains the corresponding database snapshot information.

[0013] Save the backup parameter information for this batch;

[0014] Based on the database snapshot information, the list of data files to be backed up in this batch is saved to the corresponding preset list file on the local server.

[0015] Optionally, the step of performing file compression, file transfer, and file status labeling based on the preset list file, token bucket algorithm, and preset file physical backup rules includes:

[0016] The file size of each data file in the list of data files to be backed up in the preset list file is read sequentially using the preset physical backup tool and the token bucket algorithm.

[0017] When the total size of the accumulated files exceeds the input parameter, file compression is performed to obtain the corresponding compressed file;

[0018] The compressed file is transferred to the corresponding remote storage backup server, and when the file transfer is successful, the file status of the data file located at the corresponding position in the preset list file is marked as backup successful.

[0019] The process then jumps back to the step of sequentially reading the file size of each data file in the preset list file to be backed up using the preset physical backup tool and the token bucket algorithm, until all data files in this batch have been successfully uploaded to the corresponding backup server. The preset list file is then deleted, and the corresponding data file status label information is obtained.

[0020] Optionally, the method further includes:

[0021] When the file transfer is successful, delete the corresponding compressed file and reset the counter that records the total size of the batch backup files for this round to zero.

[0022] Optionally, the file compression and file transfer based on the preset list file, the token bucket algorithm, and the preset file physical backup rules include:

[0023] Initialize the token bucket and configure the corresponding token bucket capacity information and token generation rate threshold;

[0024] Tokens are periodically added to the token bucket at a constant rate until the bucket is determined to be full based on the token bucket capacity information;

[0025] The preset physical backup tool reads the data file from the preset list file, consumes the tokens in the token bucket, and allows the corresponding data file to be output when the number of tokens in the token bucket meets the preset condition, so as to complete the corresponding file compression and transmission operations.

[0026] Optionally, the step of triggering the corresponding breakpoint resume process based on the preset list files of the previous batch, the corresponding data file status annotation information, and the database snapshot information includes:

[0027] The database snapshot information of the previous batch is subjected to a corresponding validity check process using the preset physical backup tool to obtain the corresponding check results.

[0028] If the detection results indicate that the database snapshot information from the previous batch has not expired, then the snapshot information is backed up.

[0029] Based on the corresponding data file status labeling information, starting from the file position of the data file that was not marked as successfully backed up in the preset list file of the previous batch, the file continues to be read, and corresponding file compression, file transfer, and file status labeling are performed to complete the corresponding breakpoint resume process.

[0030] Optionally, the method further includes:

[0031] If a preset network anomaly occurs during streaming backup, the backup will be terminated or a corresponding retry operation will be performed based on a preset retry strategy by identifying the type of the anomaly. The preset retry strategy includes a fixed interval retry strategy and an exponential backoff retry strategy.

[0032] Secondly, this application provides a database streaming backup and breakpoint resume device, comprising:

[0033] The file saving module is used to take a database snapshot by starting a preset physical backup tool, and save the list of data files to be backed up in this batch to the corresponding preset list file on the local server based on the obtained database snapshot information.

[0034] The streaming backup module is used to perform file compression, file transfer, and file status labeling based on the preset list file, token bucket algorithm, and preset file physical backup rules, until all data files in this batch are successfully uploaded to the corresponding backup server, complete the corresponding streaming backup operation, and obtain the corresponding data file status labeling information.

[0035] The information judgment module is used to determine whether the preset list file and the database snapshot information of the previous batch exist on the local server if the preset physical backup tool is restarted again within a preset time period, so as to obtain the corresponding judgment result.

[0036] The breakpoint resume module is used to prompt for backup type selection when the judgment result indicates that it exists. If the received user selection result is breakpoint resume, the corresponding breakpoint resume process is triggered based on the preset list files of the previous batch, the corresponding data file status label information, and the database snapshot information to complete the corresponding file backup.

[0037] Thirdly, this application provides an electronic device, comprising:

[0038] Memory, used to store computer programs;

[0039] A processor is configured to execute the computer program to implement the steps of the aforementioned method for resuming interrupted data transfer during database streaming backup.

[0040] Fourthly, this application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the steps of the aforementioned database streaming backup breakpoint resume method.

[0041] As can be seen, in this application, a preset physical backup tool is launched to take a database snapshot, and based on the obtained database snapshot information, the list of data files to be backed up in this batch is saved to the corresponding preset list file on the local server. Based on the preset list file, the token bucket algorithm, and the preset file physical backup rules, file compression, file transfer, and file status labeling are performed until all data files in this batch are successfully uploaded to the corresponding backup server, completing the corresponding streaming backup operation and obtaining the corresponding data file status label information. If the preset physical backup tool is launched again within a preset time period, it is determined whether the preset list file and the database snapshot information of the previous batch exist on the local server to obtain the corresponding judgment result. When the judgment result indicates that they exist, a backup type selection prompt is given. If the received user selection result is breakpoint resume, the corresponding breakpoint resume process is triggered based on the preset list file of the previous batch, the corresponding data file status label information, and the database snapshot information to complete the corresponding file backup. In other words, the method of recording database snapshot information and backing up a list of data files to a corresponding preset list file in this application is a prerequisite for enabling breakpoint resumption of database backup data. Furthermore, by updating the file status of the data files in the preset list file after successful data transmission, and utilizing this status information during breakpoint resumption, the backup data can be resumed. This effectively avoids duplicate transmission of backup data, improves database backup efficiency, and reduces the impact on database server storage space. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0043] Figure 1 A flowchart of a database streaming backup and breakpoint resume method provided in this application;

[0044] Figure 2 A schematic diagram of a database streaming backup process provided in this application;

[0045] Figure 3 A flowchart illustrating a token bucket algorithm provided in this application;

[0046] Figure 4 A schematic diagram illustrating a database resume download process provided in this application;

[0047] Figure 5A schematic diagram of a database streaming backup and breakpoint resume device provided in this application;

[0048] Figure 6 This application provides a structural diagram of an electronic device. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Currently, a common method for database disaster recovery is to regularly back up the data stored in the database. However, due to the massive amount of business data accumulated daily, database backups can be time-consuming, often taking hours, and placing significant pressure on network transmission during the process. Furthermore, the long backup process and high network transmission pressure can lead to network interruptions during backup. Conventional backup methods can only restart the backup process by re-initiating the backup command, requiring the retransmission of the backup data from the previous interruption, which creates unnecessary network transmission pressure and generates redundant backup data on the server hosting the backup data. Therefore, this application provides a database streaming backup solution with breakpoint resumption, which effectively avoids the repeated transmission of backup data, improves database backup efficiency, and reduces the impact on database server storage space.

[0051] See Figure 1 As shown in the figure, this invention discloses a method for resuming interrupted data transfer during database streaming backup, comprising:

[0052] Step S11: Start the preset physical backup tool to take a database snapshot, and save the list of data files to be backed up in this batch to the corresponding preset list file on the local server based on the obtained database snapshot information.

[0053] Combination Figure 2 As shown in this embodiment, it is important to understand that a preset physical backup tool needs to be started on the machine where the database service is running. The startup parameters of the corresponding backup tool mainly include the database administrator account information, data directory, and maximum size of batch backup files.

[0054] Subsequently, the preset physical backup tool connects to the database service and begins the data backup process. First, it executes a database snapshot command to save the database snapshot information for this backup. The database snapshot information contains a snapshot of the data storage in the database at the time of this backup. It also stores the backup parameter information used in this backup. For database-level snapshots, the complete state of database data files, log files, control files, etc., at a specific moment is captured, including all files and directory structures under the data directory. This typically involves copying or referencing the metadata and data blocks of the data files, and also includes the final transaction execution state of the log files, including the last saved transaction number information. That is, the step of launching a preset physical backup tool, taking a database snapshot, and saving the list of data files to be backed up in this batch to the corresponding preset list file on the local server based on the obtained database snapshot information includes: launching the preset physical backup tool based on the corresponding tool launch parameters; wherein, the tool launch parameters include the corresponding database administrator account information and input parameters used to characterize the batch backup file size threshold; capturing the complete state of the database data files, log files, and control files at any given time through the preset physical backup tool, taking a database snapshot, and obtaining the corresponding database snapshot information; saving the backup parameter information for this batch; and saving the list of data files to be backed up in this batch to the corresponding preset list file on the local server based on the database snapshot information.

[0055] It's important to understand that during the process of saving files to a preset list file using the preset physical backup tool, the backup tool needs to open a list file that records the status of data files during the backup process. Based on the database snapshot information, it outputs a list of all data files that need to be backed up in this backup to this list file. The header of this list file stores the backup execution time information, and when the next backup is executed, it will first check the time information contained in all snapshot files to delete expired snapshot files, preventing too many snapshot files from consuming too much space.

[0056] Step S12: Based on the preset list file, token bucket algorithm, and preset file physical backup rules, perform file compression, file transfer, and file status labeling until all data files in this batch are successfully uploaded to the corresponding backup server, complete the corresponding streaming backup operation, and obtain the corresponding data file status labeling information.

[0057] Specifically, in combination Figure 2As shown, in this embodiment, the file compression, file transfer, and file status labeling based on the preset list file, token bucket algorithm, and preset file physical backup rules include: sequentially reading the file size of each data file in the preset list file to be backed up using the preset physical backup tool and token bucket algorithm; when the total file size exceeds the input parameter, performing file compression to obtain the corresponding compressed file; transferring the compressed file to the corresponding remote storage backup server, and when the file transfer is successful, labeling the file status of the data file located at the corresponding position in the preset list file as successfully backed up; returning to the step of sequentially reading the file size of each data file in the preset list file to be backed up using the preset physical backup tool and token bucket algorithm, until all data files in this batch have been successfully uploaded to the corresponding backup server, deleting the preset list file, and obtaining the corresponding data file status labeling information. In other words, when the preset physical backup tool starts a physical backup operation based on the data file list, it first reads the size of each data file to be backed up, calculates and adds it to the total size of the backup files in this batch. When the accumulated backup file size exceeds the "maximum size of batch backup files" parameter passed when the tool starts, the files in this batch are compressed. This step uses the token bucket algorithm to control the transmission rate. The basic idea of ​​the token bucket algorithm is to throw "tokens" into a "bucket" at a constant rate. Once the bucket is full, no new tokens can enter. This algorithm simulates network bandwidth usage. When the number of data packets in the network exceeds the capacity of the bucket, new data packets need to wait until a new token is thrown into the bucket. The working process of the token bucket algorithm, combined with... Figure 3 As shown, the file compression and file transfer based on the preset list file, the token bucket algorithm, and the preset file physical backup rules include: initializing the token bucket and configuring the corresponding token bucket capacity information and token generation rate threshold; periodically adding tokens to the token bucket at a constant rate until the bucket is full based on the token bucket capacity information; reading the data file from the preset list file through the preset physical backup tool, consuming the tokens in the token bucket, until the number of tokens in the token bucket meets the preset conditions, and then allowing the output of the corresponding data file to complete the corresponding file compression and transfer operations.

[0058] In summary, the token bucket algorithm consists of three stages:

[0059] 1) Token generation: Tokens are periodically added to the token bucket at a certain rate (e.g., CIR / EIR). The number of tokens in the bucket continues to increase until it reaches the bucket's capacity (e.g., CBS / EBS). At this point, excess tokens are discarded. CIR (Committed Information Rate); EIR (Excess Information Rate); CBS (Committed Burst Size); EBS (Excess Burst Size).

[0060] 2) Token Consumption: Incoming data packets consume tokens from the bucket. During network transmission, different sized data packets may consume different numbers of tokens; larger data packets consume more tokens than smaller data packets.

[0061] 3) Determine if a data packet passes: If the number of tokens in the bucket is sufficient to meet the data packet's token requirements, the data packet will be output; otherwise, the data packet will be discarded.

[0062] Furthermore, this embodiment also includes: when file transfer is successful, deleting the corresponding compressed file and setting the counter recording the total size of the batch backup files to zero. It should be understood that when the compression of this batch of files is complete, the files can be transferred to the machine storing the backup files remotely. When the compressed package of this batch of files is successfully transferred, the file status in the backup database file list can be marked as backup successful. Simultaneously, the compressed file of this batch of backup files is deleted, and the counter recording the total size of this batch of backup files is set to 0. This step uses a streaming compression algorithm. Streaming compression is a compression technology capable of processing data streams in real time, particularly suitable for real-time data transmission scenarios such as data backup. Its core idea is to compress data while reading and to output compressed data at any time to ensure data real-time performance while reducing the bandwidth required for storage and transmission. Afterwards, the backup operation for subsequent files continues, repeating the aforementioned file compression, transmission, and status marking process. When the accumulated backup file size exceeds a threshold, operations such as compression, uploading, marking the data file backup success status, and deleting the compressed package are performed. This process continues until all backup files are successfully uploaded, at which point the backup is complete, and the backup data file list file is deleted. This step, following compression, can perform necessary data encryption processing according to user configuration. Streaming encryption algorithms are real-time data encryption algorithms that allow data to be encrypted in a streaming form during transmission or storage, thereby ensuring data security and privacy. Streaming encryption algorithms are based on symmetric encryption algorithms, where both encryption and decryption use the same pseudo-random encrypted data stream (pseudo-random stream) as the key. When plaintext data passes through the encryption algorithm, it is sequentially encrypted with the pseudo-random stream, generating a ciphertext data stream. This encryption method ensures that each encrypted data block is independent of the previous encryption operation, achieving real-time data encryption.

[0063] It's important to understand that if a network anomaly occurs during upload, causing the upload to fail, the backup will fail. However, the backup data file list file will not be deleted. Various errors can occur during network transmission, such as timeouts, connection interruptions, or data corruption. This method provides a robust error handling mechanism. When an error occurs, the method identifies the error type and decides whether to retry the current block or terminate the backup. Retry strategies can include fixed-interval retries, exponential backoff retries, etc., to balance transmission efficiency and network load. Specifically, it also includes: if a preset network anomaly occurs during streaming backup, the method identifies the type of the anomaly and terminates the backup or performs a corresponding retry operation based on the preset retry strategy; wherein the preset retry strategy includes corresponding fixed-interval retry strategies and exponential backoff retries.

[0064] Step S13: If the preset physical backup tool is restarted again within a preset time period, determine whether the preset list file and the database snapshot information of the previous batch exist on the local server to obtain the corresponding judgment result.

[0065] In this embodiment, combined with Figure 4 As shown, when a user initiates a backup operation again within a short period of time, the preset physical backup tool will first check if the backup data file list file and database snapshot information generated during the last backup exist. If they exist, the user will be prompted whether to perform a resume operation based on the last backup, or to perform a completely new backup operation. Before using the database snapshot, a validity check will be performed. The main reasons for snapshot invalidation are:

[0066] 1) Source database corruption: If the source database is corrupted for any reason (such as hardware failure, software error, or malicious attack), all its related database snapshots will also become unavailable. This is because database snapshots are read-only static views of the source database, and they depend on the integrity and correctness of the source database.

[0067] 2) Storage issues: Database snapshots require sufficient storage space to store their data. If the storage device lacks sufficient space or malfunctions, the database snapshot may become unusable.

[0068] 3) Incorrect configuration or management: Improper operation during database snapshot configuration or management can lead to snapshot problems. For example, if the regular backup policy is not set correctly, or if there is insufficient available space in the storage path of the backup files, snapshots may fail.

[0069] 4) Snapshot Expiration: Database snapshots are static; they reflect the database state at the time the snapshot is created. Over time, the data in a snapshot may become outdated or irrelevant, and therefore, in some cases, it may be intentionally made unavailable.

[0070] 5) System and application upgrades: When the operating system, database system, or application is upgraded, existing database snapshots may become incompatible or fail to function correctly. This is because upgrades may introduce new features, fix bugs, or change system behavior, which can affect the correctness and availability of snapshots.

[0071] 6) Corrupted snapshot file: The snapshot file itself may be corrupted due to various reasons (such as file system errors, disk failures, etc.), making the snapshot unusable.

[0072] To avoid the aforementioned snapshot expiration issue, after locating the snapshot file, its availability is ensured by verifying its integrity and assessing whether the current database state corresponds to the snapshot. If the snapshot has expired, subsequent operations cannot be performed.

[0073] Step S14: When the judgment result indicates that the backup type selection prompt is given, if the received user selection result is breakpoint resume, the corresponding breakpoint resume process is triggered based on the preset list files of the previous batch, the corresponding data file status label information and the database snapshot information to complete the corresponding file backup.

[0074] Specifically, in this embodiment, when the user chooses to continue the backup based on the previous backup, the backup tool first reads the database snapshot information saved in the previous backup, and starts the backup based on the database section position saved in the snapshot information and the backup data file list file. The default physical backup tool will directly skip files marked as successfully backed up in the backup data file list, and start reading data files from the position of files that have not yet been successfully backed up, repeating the streaming backup process shown in S12. During the transmission, both the client and the server need to record the information of files that have been successfully transmitted. This is achieved by maintaining a breakpoint record table or log. This method avoids inconsistencies between the backup information recorded by the client and the server; when inconsistencies occur, the information recorded by the server prevails. In other words, the process of triggering the corresponding breakpoint resume transfer based on the preset list files of the previous batch, the corresponding data file status annotation information, and the database snapshot information includes: performing a validity check on the database snapshot information of the previous batch using the preset physical backup tool to obtain the corresponding check result; when the check result shows that the database snapshot information of the previous batch is not invalid, performing snapshot information backup; and based on the corresponding data file status annotation information, continuing to read files from the file position of the data files in the preset list files of the previous batch that were not marked as successfully backed up, and performing corresponding file compression, file transfer, and file status annotation to complete the corresponding breakpoint resume transfer process. This allows for continuous transmission during the database backup process, fully utilizing uploaded data from the previous failed backup, greatly reducing network pressure during the database backup process, and shortening the overall database backup time.

[0075] Understandably, once the breakpoint resume operation is successfully executed, the backup is complete. If a network anomaly occurs again during the breakpoint resume backup process, causing the backup to be interrupted, the breakpoint resume operation can be executed again by repeating S13 and S14 until the backup is successful.

[0076] In summary, the snapshot information recording the backup database section information and the list file of backup data files in this embodiment are key to achieving database backup breakpoint resumption. This ensures that when the database is backed up again, the starting position of the last backup can be found, guaranteeing the integrity and accuracy of the backup data after breakpoint resumption. Simultaneously, by splitting the data files to be backed up into multiple batches, with only a small number of data files compressed in each batch, it ensures that only a small amount of storage space is occupied on the machine hosting the database service, while also achieving the goal of compressed data file storage, reducing the overall storage space occupied by backup files—a feature not found in traditional backup methods. Therefore, the database streaming backup breakpoint resumption method proposed in this embodiment better meets the requirements of not occupying a large amount of database server storage space during physical database backup, small backup file size, and resuming transmission in case of abnormal interruption during the backup process compared to traditional methods.

[0077] Therefore, in this embodiment, a preset physical backup tool is activated to take a database snapshot. Based on the obtained database snapshot information, the list of data files to be backed up in this batch is saved to the corresponding preset list file on the local server. Based on the preset list file, the token bucket algorithm, and the preset physical backup rules, file compression, file transfer, and file status labeling are performed until all data files in this batch are successfully uploaded to the corresponding backup server, completing the corresponding streaming backup operation and obtaining the corresponding data file status label information. If the preset physical backup tool is activated again within a preset time period, it is determined whether the preset list file and the database snapshot information of the previous batch exist on the local server to obtain the corresponding judgment result. When the judgment result indicates that they exist, a backup type selection prompt is given. If the received user selection result is breakpoint resume, the corresponding breakpoint resume process is triggered based on the preset list file of the previous batch, the corresponding data file status label information, and the database snapshot information to complete the corresponding file backup. In other words, the method of recording database snapshot information and backing up a list of data files to a corresponding preset list file in this application is a prerequisite for enabling breakpoint resumption of database backup data. Furthermore, by updating the file status of the data files in the preset list file after successful data transmission, and utilizing this status information during breakpoint resumption, the backup data can be resumed. This effectively avoids duplicate transmission of backup data, improves database backup efficiency, and reduces the impact on database server storage space.

[0078] See Figure 5 As shown in the illustration, this application also discloses a database streaming backup interruption resumption device, comprising:

[0079] The file saving module 11 is used to take a database snapshot by starting a preset physical backup tool, and save the list of data files to be backed up in this batch to the corresponding preset list file on the local server based on the obtained database snapshot information.

[0080] The streaming backup module 12 is used to perform file compression, file transfer, and file status labeling based on the preset list file, token bucket algorithm, and preset file physical backup rules, until all data files in this batch are successfully uploaded to the corresponding backup server, complete the corresponding streaming backup operation, and obtain the corresponding data file status labeling information.

[0081] The information judgment module 13 is used to determine whether the preset list file and the database snapshot information of the previous batch exist on the local server if the preset physical backup tool is restarted again within a preset time period, so as to obtain the corresponding judgment result.

[0082] The breakpoint resume module 14 is used to prompt for backup type selection when the judgment result shows that it exists. If the received user selection result is breakpoint resume, the corresponding breakpoint resume process is triggered based on the preset list files of the previous batch, the corresponding data file status label information and the database snapshot information to complete the corresponding file backup.

[0083] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0084] Therefore, it is evident that the method of recording database snapshot information and backing up the data file list to the corresponding preset list file in this application is a prerequisite for realizing the interrupted resumption of database backup data. Furthermore, by updating the file status of the data files in the preset list file after successful data transmission, and utilizing this status information during interrupted resumption, the backup data can be resumed. This effectively avoids duplicate transmission of backup data, improves database backup efficiency, and reduces the impact on database server storage space.

[0085] In some specific embodiments, the file saving module 11 can be used to: launch a preset physical backup tool based on corresponding tool startup parameters; wherein, the tool startup parameters include corresponding database administrator account information and input parameters used to characterize the batch backup file size threshold; capture the complete state of database data files, log files, and control files at any given time through the preset physical backup tool, perform a database snapshot, and obtain corresponding database snapshot information; save the backup parameter information for this batch; and save the list of data files to be backed up in this batch to a corresponding preset list file on the local server based on the database snapshot information.

[0086] In some specific embodiments, the streaming backup module 12 can be used to: sequentially read the file size of each data file in the preset list of data files to be backed up using the preset physical backup tool and the token bucket algorithm; when the total file size is greater than the input parameter, compress the file to obtain the corresponding compressed file; transmit the compressed file to the corresponding remote storage backup server, and when the file transmission is successful, mark the file status of the data file located at the corresponding position in the preset list as backed up successfully; jump back to the step of sequentially reading the file size of each data file in the preset list of data files to be backed up using the preset physical backup tool and the token bucket algorithm, until all data files in this batch have been successfully uploaded to the corresponding backup server, delete the preset list file, and obtain the corresponding data file status marking information.

[0087] In some specific embodiments, the streaming backup module 12 can be used to: initialize the token bucket and configure the corresponding token bucket capacity information and token generation rate threshold; periodically add tokens to the token bucket at a constant rate until the bucket is full based on the token bucket capacity information; read the data file in the preset list file through the preset physical backup tool, consume the tokens in the token bucket, and allow the output of the corresponding data file when the number of tokens in the token bucket meets the preset conditions, so as to complete the corresponding file compression and transmission operations.

[0088] In some specific embodiments, the database streaming backup breakpoint resume device can also be used to: delete the corresponding compressed file when the file transfer is successful, and reset the counter that records the total size of the batch backup files in this round to zero.

[0089] In some specific embodiments, the breakpoint resume module 14 can be used to: perform a corresponding validity check process on the database snapshot information of the previous batch using the preset physical backup tool to obtain the corresponding check result; when the check result shows that the database snapshot information of the previous batch is not invalid, perform snapshot information backup; based on the corresponding data file status labeling information, continue reading files from the file position of the data files in the preset list file of the previous batch that are not marked as successfully backed up, and perform corresponding file compression, file transfer, and file status labeling to complete the corresponding breakpoint resume process.

[0090] In some specific embodiments, the database streaming backup interruption resumption device can also be used to: if a preset network anomaly occurs during the streaming backup process, terminate the backup or perform a corresponding retry operation based on a preset retry strategy by identifying the current anomaly type; wherein, the preset retry strategy includes a corresponding fixed interval retry strategy and an exponential backoff retry strategy.

[0091] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0092] Figure 6 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the database streaming backup and breakpoint resumption method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0093] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0094] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0095] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the database streaming backup breakpoint resume method disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0096] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned method for resuming interrupted data transfer during database streaming backup. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0097] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0098] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0099] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0100] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0101] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for database streaming backup with breakpoint resumption, characterized in that, include: By launching the preset physical backup tool, a database snapshot is taken, and based on the obtained database snapshot information, the list of data files to be backed up in this batch is saved to the corresponding preset list file on the local server. Based on the preset list file, token bucket algorithm, and preset file physical backup rules, file compression, file transfer, and file status labeling are performed until all data files in this batch are successfully uploaded to the corresponding backup server, completing the corresponding streaming backup operation and obtaining the corresponding data file status labeling information. If the preset physical backup tool is restarted again within a preset time period, it is determined whether the preset list file and the database snapshot information of the previous batch exist on the local server, so as to obtain the corresponding judgment result. When the judgment result indicates that a backup type selection prompt is given, if the received user selection result is "resumable from breakpoint", the corresponding resume from breakpoint process is triggered based on the preset list files of the previous batch, the corresponding data file status label information, and the database snapshot information to complete the corresponding file backup.

2. The database streaming backup breakpoint resumption method according to claim 1, characterized in that, The step of launching a preset physical backup tool to take a database snapshot, and saving the list of data files to be backed up in this batch to a corresponding preset list file on the local server based on the obtained database snapshot information, includes: The preset physical backup tool is launched based on the corresponding tool launch parameters; wherein, the tool launch parameters include the corresponding database administrator account information and input parameters used to characterize the batch backup file size threshold; The preset physical backup tool captures the complete state of the database data files, log files, and control files at any given moment, performs a database snapshot, and obtains the corresponding database snapshot information. Save the backup parameter information for this batch; Based on the database snapshot information, the list of data files to be backed up in this batch is saved to the corresponding preset list file on the local server.

3. The database streaming backup breakpoint resumption method according to claim 2, characterized in that, The process of file compression, file transfer, and file status labeling based on the preset list file, token bucket algorithm, and preset file physical backup rules includes: The file size of each data file in the list of data files to be backed up in the preset list file is read sequentially using the preset physical backup tool and the token bucket algorithm. When the total size of the accumulated files exceeds the input parameter, file compression is performed to obtain the corresponding compressed file; The compressed file is transferred to the corresponding remote storage backup server, and when the file transfer is successful, the file status of the data file located at the corresponding position in the preset list file is marked as backup successful. The process then jumps back to the step of sequentially reading the file size of each data file in the preset list file to be backed up using the preset physical backup tool and the token bucket algorithm, until all data files in this batch have been successfully uploaded to the corresponding backup server. The preset list file is then deleted, and the corresponding data file status label information is obtained.

4. The database streaming backup breakpoint resumption method according to claim 3, characterized in that, Also includes: When the file transfer is successful, delete the corresponding compressed file and reset the counter that records the total size of the batch backup files for this round to zero.

5. The database streaming backup breakpoint resumption method according to claim 3, characterized in that, The file compression and file transfer based on the preset list file, token bucket algorithm, and preset file physical backup rules include: Initialize the token bucket and configure the corresponding token bucket capacity information and token generation rate threshold; Tokens are periodically added to the token bucket at a constant rate until the bucket is determined to be full based on the token bucket capacity information; The preset physical backup tool reads the data file from the preset list file, consumes the tokens in the token bucket, and allows the corresponding data file to be output when the number of tokens in the token bucket meets the preset condition, so as to complete the corresponding file compression and transmission operations.

6. The method for database streaming backup with breakpoint resumption according to claim 1, characterized in that, The process of triggering a breakpoint resume based on the preset list files of the previous batch, the corresponding data file status annotation information, and the database snapshot information includes: The database snapshot information of the previous batch is subjected to a corresponding validity check process using the preset physical backup tool to obtain the corresponding check results. If the detection results indicate that the database snapshot information from the previous batch has not expired, then the snapshot information is backed up. Based on the corresponding data file status labeling information, starting from the file position of the data file that was not marked as successfully backed up in the preset list file of the previous batch, the file continues to be read, and corresponding file compression, file transfer, and file status labeling are performed to complete the corresponding breakpoint resume process.

7. The method for database streaming backup with breakpoint resumption according to any one of claims 1 to 6, characterized in that, Also includes: If a preset network anomaly occurs during streaming backup, the backup will be terminated or a corresponding retry operation will be performed based on a preset retry strategy by identifying the type of the anomaly. The preset retry strategy includes a fixed interval retry strategy and an exponential backoff retry strategy.

8. A device for database streaming backup with breakpoint resumption, characterized in that, include: The file saving module is used to take a database snapshot by starting a preset physical backup tool, and save the list of data files to be backed up in this batch to the corresponding preset list file on the local server based on the obtained database snapshot information. The streaming backup module is used to perform file compression, file transfer, and file status labeling based on the preset list file, token bucket algorithm, and preset file physical backup rules, until all data files in this batch are successfully uploaded to the corresponding backup server, complete the corresponding streaming backup operation, and obtain the corresponding data file status labeling information. The information judgment module is used to determine whether the preset list file and the database snapshot information of the previous batch exist on the local server if the preset physical backup tool is restarted again within a preset time period, so as to obtain the corresponding judgment result. The breakpoint resume module is used to prompt for backup type selection when the judgment result indicates that it exists. If the received user selection result is breakpoint resume, the corresponding breakpoint resume process is triggered based on the preset list files of the previous batch, the corresponding data file status label information, and the database snapshot information to complete the corresponding file backup.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the database streaming backup breakpoint resume method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs, which, when executed by a processor, implement the database streaming backup breakpoint resume method as described in any one of claims 1 to 7.

Images