Data backup method and device, equipment, storage medium and program product

Abstract

The invention relates to a data backup method and device, equipment, a storage medium and a program product. The method comprises the following steps: before executing data deletion processing, detecting whether a data slice corresponding to a fingerprint marked as a to-be-repaired state meets a data integrity condition or not; if at least one data slice corresponding to the fingerprint marked as the to-be-repaired state does not meet the data integrity condition, switching the data backup mode into a full backup mode; if the data slices corresponding to the fingerprints marked as the to-be-repaired state all meet the data integrity condition, switching the data backup mode into a first backup mode; wherein in the first backup mode, incremental backup of preset backup times is executed; and in the data backup process, performing backup processing on the target data according to the data backup mode. By adopting the method, the data backup flexibility can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data backup method, device, equipment, storage medium and program product. Background Art

[0002] With the rapid development of Internet technology, various types of data are increasing, and users are paying more and more attention to data protection. In general data backup systems, users have different data backup needs according to different scenarios. Existing mature backup software is basically based on various means of incremental backup to provide high-speed backup mode with high deduplication rate and low space occupation for high-frequency data backup scenarios.

[0003] However, in this mode, if data is accidentally deleted or damaged, it will have a chain reaction effect on a large number of backup copies. Therefore, data backup flexibility is low in this backup mode, resulting in a high risk of data loss. Summary of the invention

[0004] Based on this, it is necessary to provide a data backup method, device, equipment, storage medium and program product that can improve the flexibility of data backup in response to the above technical problems.

[0005] In a first aspect, the present application provides a data backup method. The method comprises:

[0006] Before executing the data deletion process, check whether the data slice corresponding to the fingerprint marked as a to-be-repaired state meets the data integrity condition;

[0007] If there is at least one data slice corresponding to a fingerprint marked as being in a state of being repaired that does not meet the data integrity condition, the data backup mode is switched to the full backup mode;

[0008] If the data slices corresponding to the fingerprints marked as being in a state to be repaired all meet the data integrity condition, the data backup mode is switched to the first backup mode; wherein, in the first backup mode, incremental backups of a preset number of backup times are performed;

[0009] During the data backup process, backup processing is performed on the target data according to the data backup mode.

[0010] In one embodiment, when the data backup mode is the first backup mode, performing backup processing on the target data according to the data backup mode includes:

[0011] Determine whether this is the first backup process for the target data; if not, determine the number of incremental backups that have been performed; if it is determined that the number of incremental backups that have been performed is less than the preset number of backups, perform incremental backup processing on the target data.

[0012] In one embodiment, the method further comprises:

[0013] When it is determined that the number of incremental backups performed is not less than the preset number of backups, a partial full backup process is performed on the target data; the process of the partial full backup process includes: performing slicing processing on the target data to obtain multiple first data slices, and generating a first fingerprint for each first data slice; comparing each first fingerprint with each second fingerprint available in the fingerprint library, determining a target first fingerprint that does not exist in the fingerprint library, and using the target first fingerprint as a newly added fingerprint; storing the newly added fingerprint and the first data slice corresponding to the first newly added fingerprint, and generating a first snapshot of the target data in the current backup stage.

[0014] In one embodiment, performing incremental backup processing on target data includes:

[0015] Generate a temporary snapshot of the target data, and obtain a historical snapshot generated when the target data was last backed up; compare the temporary snapshot with the historical snapshot to obtain a data change path; determine the newly added data in the target data according to the data change path; perform data slicing processing on the newly added data to obtain multiple second data slices, and generate a second fingerprint of each second data slice; store the second data slice and the second fingerprint.

[0016] In one of the embodiments, the method further includes: generating a second snapshot according to the historical snapshot and metadata information of the newly added data, and storing the second snapshot.

[0017] In one of the embodiments, the method further includes: determining whether the data slices corresponding to the fingerprints to be deleted are referenced; if so, marking the fingerprints corresponding to the referenced data slices as being in a state to be repaired.

[0018] In a second aspect, the present application also provides a data backup device. The device comprises:

[0019] A detection module, used for detecting whether the data slice corresponding to the fingerprint marked as being in a state to be repaired meets the data integrity condition before executing the data deletion process;

[0020] A first switching module, configured to switch the data backup mode to a full backup mode if there is at least one data slice corresponding to a fingerprint marked as being in a state to be repaired that does not meet a data integrity condition;

[0021] A second switching module is used to switch the data backup mode to the first backup mode if the data slices corresponding to the fingerprints marked as being in a state to be repaired all meet the data integrity condition; wherein, in the first backup mode, incremental backup of a preset number of backup times is performed;

[0022] The execution module is used to perform backup processing on the target data according to the data backup mode during the data backup process.

[0023] In one embodiment, when the data backup mode is the first backup mode, the execution module is specifically configured to include:

[0024] Determine whether this is the first backup process for the target data; if not, determine the number of incremental backups that have been performed; if it is determined that the number of incremental backups that have been performed is less than the preset number of backups, perform incremental backup processing on the target data.

[0025] In one of the embodiments, the device further includes a local backup module, configured to:

[0026] When it is determined that the number of incremental backups performed is not less than the preset number of backups, a partial full backup process is performed on the target data; the process of the partial full backup process includes: performing slicing processing on the target data to obtain multiple first data slices, and generating a first fingerprint for each first data slice; comparing each first fingerprint with each second fingerprint available in the fingerprint library, determining a target first fingerprint that does not exist in the fingerprint library, and using the target first fingerprint as a newly added fingerprint; storing the newly added fingerprint and the first data slice corresponding to the first newly added fingerprint, and generating a first snapshot of the target data in the current backup stage.

[0027] In one embodiment, the execution module is specifically configured to:

[0028] Generate a temporary snapshot of the target data, and obtain a historical snapshot generated when the target data was last backed up; compare the temporary snapshot with the historical snapshot to obtain a data change path; determine the newly added data in the target data according to the data change path; perform data slicing processing on the newly added data to obtain multiple second data slices, and generate a second fingerprint of each second data slice; store the second data slice and the second fingerprint.

[0029] In one of the embodiments, the device further includes a snapshot module, which is used to: generate a second snapshot according to the historical snapshot and metadata information of the newly added data, and store the second snapshot.

[0030] In one of the embodiments, the device further includes a marking module, which is used to: determine whether the data slices corresponding to each fingerprint to be deleted are referenced; if so, mark the fingerprint corresponding to the referenced data slice as a state to be repaired.

[0031] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0032] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect above.

[0033] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0034] The above-mentioned data backup method, device, equipment, storage medium and program product detect whether the data slice corresponding to the fingerprint marked as the state to be repaired meets the data integrity condition before executing the data deletion process; if there is at least one data slice corresponding to the fingerprint marked as the state to be repaired that does not meet the data integrity condition, the data backup mode is switched to the full backup mode; if the data slices corresponding to the fingerprint marked as the state to be repaired all meet the data integrity condition, the data backup mode is switched to the first backup mode; wherein, in the first backup mode, incremental backup of a preset number of backups is performed; during the data backup process, backup processing is performed on the target data according to the data backup mode. In this way, by executing different data backup modes, the flexibility of data backup can be improved and the integrity of data slices can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A schematic diagram of a data backup method in one embodiment;

[0037] Figure 2 is a schematic diagram of a snapshot corresponding to target data in an embodiment;

[0038] Figure 3 is a schematic diagram of fingerprint partitioning in one embodiment;

[0039] Figure 4 A schematic diagram of full data backup in one embodiment;

[0040] Figure 5 A schematic diagram of a process of backing up target data in one embodiment;

[0041] Figure 6 A schematic diagram of the process of incremental backup processing in one embodiment;

[0042] Figure 7 A schematic diagram of data path comparison in an embodiment;

[0043] Figure 8 is a schematic diagram of a snapshot chain in one embodiment;

[0044] Fig. 9 A schematic diagram of a partial full backup process in one embodiment;

[0045] Fig.10 is another snapshot chain schematic diagram in one embodiment;

[0046] Fig.11 A schematic diagram of a backup mode switching process in an embodiment;

[0047] Fig.12 A schematic diagram of a data backup process in an incremental backup and a partial full backup process in one embodiment;

[0048] Fig.13 is a structural block diagram of a data backup device in one embodiment;

[0049] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It should be understood that many specific details are set forth in the following description to facilitate a full understanding of the present application, but the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0052] With the rapid development of Internet technology, enterprises are paying more and more attention to data protection. Data processing and protection have become an indispensable part of enterprise operations, because data not only represents the core competitiveness of enterprises, but also the cornerstone of their sustainable development.

[0053] In the field of data protection, general backup systems play a vital role, and incremental backup, as an efficient data backup method, is widely adopted. It greatly improves the efficiency of data backup and reduces storage costs by backing up only the data blocks that have changed since the last backup. However, an efficient incremental backup system is also accompanied by a potential problem, which is the risk of data deletion caused by a high deduplication rate. Taking the cloud backup service based on delayed deletion as an example, it uses a lock-free method to cleverly solve the timing problem of concurrent execution of data deletion and data backup, ensuring the smoothness of the backup process and the security of data. However, even such an advanced system faces some risks and challenges.

[0054] During the data deletion process, whether it is the third-party storage service or the backup system itself, there is a theoretical risk of accidental data deletion or data corruption. Once this risk occurs, due to the incremental backup mechanism, it will cause a chain reaction, making all snapshots based on the same snapshot link unavailable. This situation is particularly serious in the scenario of continuous and high-frequency data backup, because after a failure occurs, all new backup snapshots generated by incremental backup will be unavailable, which not only wastes system resources, but also may cause the enterprise to face the serious problem of no available snapshots for the latest data backup. In order to meet this challenge, there is currently a lack of a means to automatically reduce losses in a timely manner and repair data as automatically as possible.

[0055] In general data backup systems, users have different data backup requirements according to different scenarios, some of which require high-frequency backup of hot data, and some require long-term backup of cold data. For high-frequency data backup scenarios, existing mature backup software basically provides a high-speed backup mode with high deduplication rate and low space occupation through various means of incremental backup. Taking the cloud backup service based on delayed deletion as an example, although its incremental backup and delayed deletion features bring efficient data backup and relatively safe data deletion features, the following problems still exist:

[0056] 1. In the incremental backup mode, the backup snapshot link is complex and indispensable, and is highly dependent on the data integrity of a certain incremental backup or local data. It has weak risk resistance. When data is accidentally deleted or damaged, it will have a chain effect on a large number of backup copies. An effective means of data isolation and timely loss prevention is required.

[0057] 2. Although under multiple delayed deletion strategies, the data life cycle is long enough, and the delayed deletion feature can be relied upon to automatically identify accidentally deleted data, GC (Garbage Collection) can be stopped when data is accidentally deleted, and then manual intervention can be made to retrieve data through the trash can. However, this stage is not only manpower-consuming, but if the processing window is too long, users will intuitively feel the failure of the data copy, which does not achieve the purpose of stopping losses in a timely manner, and the system reliability still needs to be improved.

[0058] In view of this, considering that data deletion and data backup and recovery are closely related to each other, and relying on the nature of continuous data backup of the backup system itself, the embodiment of the present application provides a data backup method, which flexibly intersperses full or partial backup efforts during the continuous backup process, and utilizes the backup characteristics of these backup methods themselves to take into account data security and data processing efficiency as much as possible. By introducing flexible frequency incremental backup, partial full backup and triggered full backup to cooperate with delayed deletion, automatic correction and automatic repair of data in continuous data backup are achieved, and the scope of loss when data is accidentally deleted or damaged is minimized as much as possible. At the same time, the data is automatically corrected through the most recent backup action as much as possible, so that there is a latest available copy. The expected effect of this method in frequent backup scenarios is significant.

[0059] It should be noted that the data backup method provided in the embodiment of the present application may be implemented as a data backup device, which may be implemented as part or all of a computer device through software, hardware, or a combination of software and hardware. The computer device may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, and servers, etc. The Internet of Things devices may be smart TVs, smart car devices, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers. The following is an example of the application of this method to a computer device.

[0060] Before describing the data backup method, the technology involved in the embodiments of the present application is first explained.

[0061] Data fingerprint FP (Fingerprint): Based on the requirements of different backup scenarios, a unique identifier is generated by using a suitable algorithm for the hash value, metadata, or other characteristic information of a data block, which is used to index and manage backup files.

[0062] Incremental Backup: Compared with the last backup, this backup only backs up the changed data, so it has the advantages of efficient and high-speed backup. At the same time, because it is necessary to obtain complex backup sets during recovery, a certain recovery efficiency will be sacrificed, and it must rely heavily on all backup sets to be normal, otherwise all will be damaged.

[0063] Full Backup: A technology that backs up all data without regard to previous backups. Because all data needs to be backed up, the backup efficiency is low, but it has higher data security and higher recovery efficiency.

[0064] Deferred Deletion: The data to be deleted is not processed immediately, but is processed after a delay of a period of time through specific algorithms and rules. It is often used to handle concurrent backup and recovery and data GC (garbage collection) timing issues, and provides a GC strategy in a lock-free state.

[0065] In one embodiment, Figure 1 As shown, a data backup method is provided, comprising the following steps:

[0066] Step 101, before executing the data deletion process, it is detected whether the data slice corresponding to the fingerprint marked as being in a state to be repaired meets the data integrity condition.

[0067] When each data is stored, it needs to be sliced ​​to obtain multiple data slices, and a fingerprint corresponding to each data slice is generated, and the fingerprint and the data slice are stored. That is, the fingerprint is the unique identifier of the data slice.

[0068] Optionally, a fingerprint library is deployed in the computer device, and the slices corresponding to the fingerprint are stored in the fingerprint library. Alternatively, the fingerprint is stored in the fingerprint library, and the data slices are stored in other memory spaces, and a one-to-one correspondence between the two is retained.

[0069] Optionally, the target data of the computer device backup may be a directory data or a file. The directory data may include subdirectories and / or subfiles.

[0070] It is understandable that a plurality of data may be stored in a computer device, so there are a plurality of data slices and a plurality of data fingerprints. Also, in some cases, a data slice may be deleted after being used.

[0071] Therefore, the fingerprint library includes available fingerprints corresponding to some available data slices and unavailable fingerprints corresponding to some data slices to be deleted or have been deleted.

[0072] In the embodiment of the present application, some unusable fingerprints may be marked as being in a state to be repaired. Optionally, when deleting these unusable fingerprints, the computer device determines whether there are tasks that need to reference data slices corresponding to these unusable fingerprints during execution. If so, these unusable fingerprints are marked as being in a state to be repaired.

[0073] When performing data deletion processing, since these fingerprints are marked as being in a state to be repaired, it means that the data slices corresponding to these fingerprints need to be used. It is necessary to detect whether the data slices corresponding to the fingerprints marked as being in a state to be repaired meet the data integrity conditions, so as to use different backup methods to restore the data slices corresponding to these fingerprints in a state to be repaired according to different detection results.

[0074] In addition, when the computer device backs up the target data, it generates a snapshot of the latest backup target data after each backup, so that when an application failure or file damage occurs in the storage device, the snapshot can be used to quickly restore the data to a state at a certain available time point. Take the target data as directory data as an example, Figure 2 A schematic diagram of a snapshot corresponding to target data is shown as an example. Wherein, dir is a directory, file is a file, and both are nodes in the target data. mode is a node in the snapshot. It can be understood that since the target data is a tree structure, a snapshot of the same tree structure can be formed. Wherein, the snapshot includes the metadata of each node in the target data and the association relationship between each node.

[0075] In an optional implementation, to address the timing challenges of concurrent access to an index, delayed deletion adopts a strategy of dividing the index into three different categories: active, forget, and delete. This classification method can effectively handle concurrent timing issues without introducing additional locking mechanisms. For example, Figure 3 A schematic diagram of fingerprint partitioning is shown. Index01 to index14 are fingerprints of data slices corresponding to available snapshots, which are placed in the active class. Index34 to index44 are fingerprints of data slices corresponding to deleted snapshots, which are placed in the discarded class. Chunk is a byte stream space in Chinese. These data slices can be written into the chunk for storage.

[0076] Exemplarily, the active class is used to store newly backed-up snapshots and available snapshots. Each time a backup is performed, the newly added data slices and their corresponding fingerprints are also placed in the active class, that is, any newly added fingerprints are directly updated to this class. The To-be-Cleaned class is referenced and operated only by the garbage collection (GC) process. During the GC process, expired snapshots are checked, and the fingerprints unique to these snapshots are filtered out from the active class and moved to the To-be-Cleaned class. At the same time, the transfer timestamps of these indexes or the status of the related backup tasks are recorded. In the next round of GC, if the backup tasks associated with the To-be-Cleaned class that was previously transferred to the To-be-Cleaned class have been completed, and all current snapshots no longer reference these indexes, they will be transferred to the Abandoned class; if there are still references, they will be reactivated and moved back to the Active class. The fingerprints in the Abandoned class are determined to no longer be referenced by any available snapshots, so they can be safely deleted.

[0077] Based on the above mechanism, during the delayed deletion process, snapshots can be traversed to verify data integrity, and fingerprints in the abandoned area can be checked to see if they are referenced by active snapshots. If so, it means that data has been accidentally deleted. At this time, data recovery is required.

[0078] In the embodiment of the present application, unavailable fingerprints may refer to fingerprints in the discarded class, and fingerprints in the state of being repaired are located in the discarded class of the fingerprint library. When some data slices referenced by available snapshots are deleted due to accidental deletion of data, the fingerprints of these data slices are moved to the discarded class, and the data are marked as being in the state of being repaired. They need to be restored at the next backup.

[0079] Therefore, when executing the deletion process, the computer device needs to detect whether the data slice corresponding to the fingerprint marked as being in a state to be repaired meets the data integrity condition.

[0080] Optionally, the computer device obtains a corresponding data slice based on the fingerprint of the state to be repaired. If the corresponding data slice cannot be obtained or the obtained data slice is incomplete (for example, the data slice is incomplete due to data corruption), it is determined that the data slice corresponding to the fingerprint of the state to be repaired does not meet the data integrity condition, otherwise it meets the data integrity condition.

[0081] Step 102: If there is at least one data slice corresponding to a fingerprint marked as being in a state to be repaired that does not meet the data integrity condition, the data backup mode is switched to a full backup mode.

[0082] In the full backup mode, the target data is fully backed up, that is, the target data is completely backed up instead of only part of the data. In this way, all the data of the target data can be restored in time.

[0083] Step 103: If the data slices corresponding to the fingerprints marked as being in a state to be repaired all meet the data integrity condition, the data backup mode is switched to the first backup mode.

[0084] In the first backup mode, a preset number of incremental backups are performed. Incremental backup means that after a full backup or the last incremental backup, each subsequent backup only needs to back up the data that has been added or modified compared to the previous one.

[0085] Here, since the snapshot generated after each incremental backup depends on the snapshot generated by the previous incremental backup, the multiple snapshots generated by multiple consecutive incremental backups will form a snapshot chain and are related. In the embodiment of the present application, by performing incremental backups for a preset number of times, no incremental backup is performed after exceeding the preset number of backups, and the snapshot chain is disconnected, thereby avoiding the unavailability of all subsequent snapshots due to the damage of one of the snapshots.

[0086] Optionally, the preset number of backups is determined based on actual needs and is not specifically limited here.

[0087] Step 104: During the data backup process, backup processing is performed on the target data according to the data backup mode.

[0088] The above data backup method detects whether the data slice corresponding to the fingerprint marked as being in a state to be repaired meets the data integrity condition before executing the data deletion process; if there is at least one data slice corresponding to the fingerprint marked as being in a state to be repaired that does not meet the data integrity condition, the data backup mode is switched to the full backup mode; if the data slices corresponding to the fingerprint marked as being in a state to be repaired all meet the data integrity condition, the data backup mode is switched to the first backup mode; wherein, in the first backup mode, incremental backups of a preset number of backups are performed; during the data backup process, backup processing is performed on the target data according to the data backup mode. In this way, the flexibility of data backup can be improved and the integrity of data slices can be ensured by executing different data backup modes.

[0089] In one embodiment, when the data backup mode is a full backup mode, backup processing is performed on the target data according to the data backup mode, including: performing slicing processing on the target data to obtain multiple first data slices, and generating a first fingerprint for each first data slice; storing the first data slices and the first fingerprint.

[0090] Optionally, as mentioned above, the first fingerprint is stored in a fingerprint library, for example, put into an activity class of the fingerprint library.

[0091] Optionally, a snapshot of the target data may be generated after the backup.

[0092] In the embodiment of the present application, the fingerprint corresponding to the failed data slice is marked as a pending repair state, and then the full backup switch is turned on. The next backup in the continuous backup scenario will perform a full backup, thereby uploading all backup data and fingerprints of the target data. Figure 4 A schematic diagram of full data backup is shown. When index44 is marked as pending repair, full backup is performed at the next backup after full backup is enabled, the data slice corresponding to index44 is restored, and index44 is moved to the active class. It can be seen that under this switching mode, at most the data changes between the fault backup and the backup after the full backup switch is enabled are lost. The snapshots and target data of the backup generated thereafter are accurate and available, and the lost data can be retrieved by sorting the retained data that has been deleted late, which improves the timeliness of ensuring data integrity.

[0093] In one embodiment, the method further includes: if the data slices corresponding to the fingerprint marked as being in a state to be repaired all meet the data integrity condition, restoring the fingerprint in the state to be repaired to a usable fingerprint, that is, restoring it to the active class.

[0094] In this way, if all fingerprints to be repaired have been repaired in the historical backup process, their status will be called back to the active area to achieve the effect of data repair, and the full backup switch will be turned off, that is, the data backup mode will be switched to the first backup mode, thereby reducing client resource loss. In addition, the efficient and high-performance incremental backup mechanism will continue to be used, and the full backup switch will be used as a backup mechanism for data recovery, taking into account both risk resistance and high backup performance.

[0095] In one embodiment, the method further includes: determining whether the data slices corresponding to the fingerprints to be deleted are referenced; if so, marking the fingerprints corresponding to the referenced data slices as being in a state to be repaired.

[0096] In this way, it is possible to effectively and quickly determine which fingerprints need to be marked as being in a state to be repaired.

[0097] In one embodiment, Figure 5 A schematic diagram of a process for backing up target data is shown. When the data backup mode is the first backup mode, the target data is backed up according to the data backup mode, including:

[0098] Step 501, determining whether the current process is the first backup process for the target data.

[0099] As mentioned above, after each backup, a snapshot of the target data after the backup can be generated. Therefore, if it is not the first backup process, there is a historical snapshot of the target data in the computer device. On the contrary, if there is no historical snapshot of the target data, it is determined that the current is the first backup.

[0100] Step 502: If not, determine the number of times the incremental backup has been performed.

[0101] If it is not the first backup process and is currently in the first backup mode, it means that incremental backup can be performed, and the number of times the incremental backup has been performed needs to be determined.

[0102] Step 503: When it is determined that the number of incremental backups performed is less than the preset number of backups, incremental backup processing is performed on the target data.

[0103] Optionally, the preset number of backups may have different values ​​for different backup scenarios. For example, different preset number of backups may be set for different data change frequencies and different data backup frequencies. In other words, the corresponding preset number of backups may be set in advance according to the change frequency of the target data.

[0104] The computer device is pre-deployed with a preset number of backups. Combined with the preset number of backups, in the first backup mode, it is possible to switch between incremental backup and local incremental backup, thereby isolating the snapshot chain, avoiding data dependence, and improving data reliability.

[0105] In one embodiment, the method further includes: if the current process is the first backup process for the target data, performing slicing process on the target data to obtain a plurality of first data slices, and generating a first fingerprint for each first data slice; and storing the first data slices and the first fingerprint.

[0106] In addition, the computer device also generates a first snapshot corresponding to the first backup of the target data, and stores the first snapshot.

[0107] The following is an explanation of the partial full backup and incremental backup in the first backup mode.

[0108] In one embodiment, Figure 6 A schematic diagram of an incremental backup process is shown. Incremental backup processing is performed on target data, including:

[0109] Step 601: Generate a temporary snapshot of target data, and obtain a historical snapshot generated when the target data was backed up last time.

[0110] The computer device may directly generate a temporary snapshot of the target data based on the structure in the target data.

[0111] Furthermore, since this is not the first backup and each backup generates a snapshot, there is at least one snapshot for the target data in the computer device. Here, the snapshot generated by each incremental backup depends on the historical snapshot generated by the last backup, that is, the historical snapshot generated when the target data was backed up last time is used as the parent snapshot of the snapshot of the current backup.

[0112] Step 602: Compare the temporary snapshot with the historical snapshot to obtain the data change path.

[0113] When there is new data or updated data in the target data, the location path corresponding to the new data is the data change path. Figure 7 The diagram of data path comparison is shown, where the target data is backed up at time T1 to obtain a historical snapshot. At time T2, the local data changes, and the backup is performed again. The historical snapshot at time T1 is pulled from the storage space as the parent snapshot, and the two snapshots are compared to accurately filter the files in the changed path. In this way, the next step is to slice the newly added data on the data change path.

[0114] Step 603: Determine the newly added data in the target data according to the data change path.

[0115] It is understandable that the newly added data is different from the last backup data, which can be data update or new addition.

[0116] Step 604: perform data slicing processing on the newly added data to obtain multiple second data slices, and generate second fingerprints of the second data slices; store the second data slices and the second fingerprints.

[0117] Optionally, as mentioned above, the second fingerprint is stored in the activity class area of ​​the fingerprint library.

[0118] In the embodiment of the present application, this incremental backup mode can efficiently implement the backup action. When the data changes slightly, only a small amount of slice calculation and upload of a small amount of data slices are required each time, thereby improving the efficiency of data backup.

[0119] In one embodiment, the method further includes: generating a second snapshot according to the historical snapshot and metadata information of the newly added data, and storing the second snapshot.

[0120] Exemplarily, a second snapshot is generated based on the metadata information of the historical snapshot and the newly added data, and the second snapshot is stored. At this time, on the snapshot chain, the historical snapshot is the parent snapshot of the second snapshot.

[0121] For example, Figure 8A schematic diagram of a snapshot chain is shown, where snap refers to a snapshot. However, if a data failure occurs and one of the snapshots fails, all subsequent associated snapshots will fail. For example Figure 8 Snap4 fails, causing snap5 and snap9 to be unavailable. To avoid this situation from having too much impact on data, the embodiment of the present application improves the local full backup process in the first incremental backup mode to isolate the snapshot chain.

[0122] In one embodiment, the method further includes: when it is determined that the number of incremental backups performed is not less than a preset number of backups, performing a partial full backup process on the target data.

[0123] Among them, in the first backup mode, when the number of incremental backups is reached, the parent snapshot is no longer pulled during the backup process. At this time, it is consistent with the traditional incremental backup, and all backup data is sliced ​​and compared. Since not all data is uploaded, it is easy to distinguish, so it is called partial full backup.

[0124] Please refer to Fig. 9 , which shows a schematic diagram of a partial full backup process. The partial full backup process includes:

[0125] Step 901 , performing slicing processing on target data to obtain a plurality of first data slices, and generating a first fingerprint of each first data slice.

[0126] Step 902: Compare each first fingerprint with each second fingerprint available in the fingerprint database, determine a target first fingerprint that does not exist in the fingerprint database, and use the target first fingerprint as a newly added fingerprint.

[0127] Step 903: store the newly added fingerprint and the first data slice corresponding to the first newly added fingerprint, and generate a first snapshot of the target data in the current backup phase.

[0128] Optionally, as mentioned above, the first newly added fingerprint is stored in the activity class of the fingerprint library.

[0129] The first snapshot is generated according to the metadata information of the target data, and the first snapshot is stored.

[0130] For example, Fig.10 In the snapshot chain diagram shown in the figure, in the first phase, three consecutive incremental backups generate snap1 to snap3. In the second phase, the first snapshot snap1 is generated based on the target data alone, without relying on snap3 in the first phase, thereby generating the snapshot chain snap1 to snap3 in the second phase. Similarly, the snapshot chain snap1 to snap3 is also obtained in the third phase. When the snap1 snapshot in the second phase fails, only the other snapshots in the second phase will be unavailable, while the snapshots in the first and third phases are still available. Compared with Figure 8 , isolate the snapshots to avoid a large number of snapshots being unavailable.

[0131] In the embodiment of the present application, considering that if a permanent incremental backup method is adopted, each backup snapshot is completely dependent on the integrity of the historical snapshot data, once data loss occurs in the snapshot link, all subsequent backup actions are useless, and the generated backup snapshots are also unavailable. In addition, while delayed deletion based on index classification has its backup deletion advantages, it still has deficiencies in data loss risk protection. Data deletion and data backup are independent of each other, and the data correction capabilities of the backup system itself in scenarios with trace data changes and high-frequency backups are not utilized. Therefore, a data automatic correction and repair mechanism that flexibly alternates between different intensity backups of combined incremental full backup, delayed deletion, and GC data integrity detection is proposed. By introducing flexible local incremental backup and full backup to cooperate with delayed deletion, automatic correction and automatic repair of data in continuous data backup can be achieved.

[0132] In the first backup mode, when the preset number of backups is reached, the starting snapshot is cleared. The current backup snapshot is used as the starting point of the next set of snapshots to isolate the snapshot chain and reduce the associated impact caused by the loss of a single snapshot data. In this way, the damage of a single snapshot on the original backup snapshot chain will not cause the subsequent snapshots to be unavailable. In this way, all data does not need to be sliced ​​each time, and the advantages of efficient incremental backup based on historical snapshots will not be given up. At the same time, data can be corrected to a certain extent to reduce the scope of damage.

[0133] In summary, the above data backup method, by implementing the delayed deletion strategy, combines the fingerprints and data verification of each state, and alternately performs different types of backup operations at an appropriate frequency, including incremental backup, partial full backup, and full backup. These three backup strategies show an increasing trend in time and computing resource consumption. Specifically, if you choose to perform incremental backup continuously, although you can efficiently utilize the backup performance, the security and risk resistance of the data are relatively low, and there is a lack of remedial measures. If you choose to perform partial full backup continuously, the backup efficiency is moderate, and you need to traverse all data slices each time to pull and compare the full fingerprints. Although it will consume more computing resources, it can guarantee the security and risk resistance of the data to a certain extent. If you choose to perform full backup continuously, although the backup efficiency is low, you need to pull the full fingerprint each time, and slice and transmit the entire data set, but this can ensure that the data has extremely high security. Therefore, you can flexibly intersperse backup modes of different strengths with delayed deletion data checks to achieve the purpose of efficient data backup and automatic fault repair. In this way, the advantages of delayed deletion and data backup can be effectively combined and coordinated. The system uses permanent incremental backup mode most of the time, with low slice loss and high backup performance. A small part of the time, local full backup is interspersed to do regular data isolation. In extreme scenarios, complete full backup means are enabled, sacrificing some backup performance in exchange for data security. The above cooperation makes the various processing processes more closely coordinated to improve the reliability of computer equipment in processing data and the performance of executing data backup.

[0134] For ease of understanding, the following is a complete example to illustrate the process of the method for automatic correction and repair of file backup data based on delayed deletion provided by the present application.

[0135] Please refer to Fig.11 , which shows a schematic diagram of a backup mode switching process, Fig.12 A schematic diagram of a data backup process in the process of incremental backup and partial full backup is shown.

[0136] The fingerprints of data slices in all snapshots are divided into the active area, the area to be cleared or the discarded area in the fingerprint library.

[0137] A delayed deletion process is set, and delayed deletion is performed according to the delayed deletion process strategy. Also, the system performs backup according to the currently set backup mode at a preset time according to the backup strategy.

[0138] Specifically, when the delayed deletion process is executed, all fingerprints in the abandoned area are first determined, where the data corresponding to the fingerprints in the abandoned area have been deleted or are incomplete. Then determine whether there are tasks that reference the data slices corresponding to these fingerprints. If there are no task references, the fingerprints in the abandoned area are normally deleted. On the contrary, if there are task references, the fingerprints referenced by the tasks are marked as pending repair status, which means that the data slices corresponding to these fingerprints need to be restored.

[0139] Determine whether the data slices corresponding to all fingerprints in the state of being repaired meet the integrity (i.e., they exist completely). If not, perform repair processing on the data slices corresponding to all fingerprints in the state of being repaired: that is, turn on the full backup switch so that the target data to which the data slice belongs is fully backed up at the next backup, so that the data slices corresponding to the fingerprints in the state of being repaired can be restored at the next backup.

[0140] If it is determined that the data slices corresponding to all the fingerprints in the state to be repaired have been restored, turn off the full backup switch, and change all the fingerprints in the state to be repaired from the discarded area to the active area. In addition, turn on the incremental backup and partial full backup switches.

[0141] When backing up data, first determine whether the current full backup switch is turned on. If it is turned on, perform a full backup: that is, slice and generate fingerprints for all data in the current directory, and then back up.

[0142] Otherwise, incremental backup or partial full backup is performed according to the incremental frequency. Including:

[0143] According to the backup strategy, when performing a backup on directory 1, determine whether there is a snapshot of directory 1. If not, it is the first backup, then perform slicing processing on the data in directory 1 to obtain multiple data slices, generate fingerprints corresponding to each data slice, store all data slices and fingerprints in the fingerprint library, and generate and store a snapshot of directory 1. The snapshot includes the metadata of each subdirectory or file in directory 1 and the structural relationship between each subdirectory and file.

[0144] If it is not the first backup, determine how many times the incremental backup has been performed. If the number of incremental backups performed is less than the preset number of backups, perform the incremental backup. Otherwise, perform a partial full backup.

[0145] The incremental backup includes: generating a temporary snapshot of the current directory 1, obtaining the most recent snapshot 1 of target 1, comparing snapshot 1 with the temporary snapshot, obtaining the newly added data path, treating the data in directory 1 on the newly added data path as newly added data, and performing data slicing and fingerprint generation on the newly added data. The data slices and fingerprints are stored, and snapshot 2 is generated based on the metadata information of snapshot 1 and the newly added data, and snapshot 2 is stored.

[0146] Performing a partial full backup includes: slicing all data in the current directory 1 and obtaining fingerprints, comparing them with the fingerprints of the data slices in the original directory 1 in the fingerprint library, determining the newly added fingerprints, and thus determining that the newly added fingerprints and the data corresponding to the newly added fingerprints are the newly added data slices. Then, the latest snapshot is generated based on the metadata and associations of all the data in the current directory 1, and the latest snapshot is stored. During the next incremental backup, the next snapshot is generated based on the latest snapshot. Snapshot isolation is achieved.

[0147] It can be seen that performing partial full backup at an appropriate frequency, relative to the risk of multiple snapshot linkage damage in permanent incremental backup, decouples the backup snapshots from each other, performs a small full backup after a certain amount of incremental snapshots are accumulated, corrects the backup starting point, avoids the situation where all subsequent snapshots are unavailable due to fingerprint damage and loss, isolates risks, and reduces damage. Based on delayed deletion of fingerprints and data checks in each state, incremental backup, partial full backup, and full backup are performed alternately at an appropriate frequency, and backup capabilities of different strengths are used flexibly as needed to achieve the purpose of efficient data backup and automatic remediation. Automatically perform data remediation in extremely low probability scenarios of accidental deletion or damage of data, while ensuring high backup performance while taking into account high data security and the ability to resist data loss risks.

[0148] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0149] Based on the same inventive concept, the embodiment of the present application also provides a data backup device for implementing the data backup method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data backup device embodiments provided below can refer to the limitations on the data backup method above, and will not be repeated here.

[0150] In one embodiment, Fig.13As shown, a data backup device is provided, the data backup device 1300 includes: a detection module 1301, a first switching module 1302, a second switching module 1303 and an execution module 1304, wherein:

[0151] The detection module 1301 is used to detect whether the data slice corresponding to the fingerprint marked as being in a state to be repaired meets the data integrity condition before executing the data deletion process;

[0152] A first switching module 1302 is configured to switch the data backup mode to a full backup mode if at least one data slice corresponding to a fingerprint marked as being in a to-be-repaired state does not meet a data integrity condition;

[0153] The second switching module 1303 is used to switch the data backup mode to the first backup mode if the data slices corresponding to the fingerprints marked as being in a state to be repaired all meet the data integrity condition; wherein, in the first backup mode, incremental backup of a preset number of backup times is performed;

[0154] The execution module 1304 is used to perform backup processing on the target data according to the data backup mode during the data backup process.

[0155] In one embodiment, when the data backup mode is the first backup mode, the execution module 1304 is specifically configured to include:

[0156] Determine whether this is the first backup process for the target data; if not, determine the number of incremental backups that have been performed; if it is determined that the number of incremental backups that have been performed is less than the preset number of backups, perform incremental backup processing on the target data.

[0157] In one of the embodiments, the device further includes a local backup module, configured to:

[0158] When it is determined that the number of incremental backups performed is not less than the preset number of backups, a partial full backup process is performed on the target data; the process of the partial full backup process includes: performing slicing processing on the target data to obtain multiple first data slices, and generating a first fingerprint for each first data slice; comparing each first fingerprint with each second fingerprint available in the fingerprint library, determining a target first fingerprint that does not exist in the fingerprint library, and using the target first fingerprint as a newly added fingerprint; storing the newly added fingerprint and the first data slice corresponding to the first newly added fingerprint, and generating a first snapshot of the target data in the current backup stage.

[0159] In one embodiment, the execution module 1304 is specifically configured to:

[0160] Generate a temporary snapshot of the target data, and obtain a historical snapshot generated when the target data was last backed up; compare the temporary snapshot with the historical snapshot to obtain a data change path; determine the newly added data in the target data according to the data change path; perform data slicing processing on the newly added data to obtain multiple second data slices, and generate a second fingerprint of each second data slice; store the second data slice and the second fingerprint.

[0161] In one of the embodiments, the device further includes a snapshot module, which is used to: generate a second snapshot according to the historical snapshot and metadata information of the newly added data, and store the second snapshot.

[0162] In one of the embodiments, the device further includes a marking module, which is used to: determine whether the data slices corresponding to each fingerprint to be deleted are referenced; if so, mark the fingerprint corresponding to the referenced data slice as a state to be repaired.

[0163] Each module in the above data backup device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0164] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.14 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data backup data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data backup method is implemented.

[0165] Those skilled in the art will understand that Fig.14The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0166] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0167] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0169] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0170] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data backup method, characterized in that: The method comprises: Before executing the data deletion process, check whether the data slice corresponding to the fingerprint marked as a to-be-repaired state meets the data integrity condition; If there is at least one data slice corresponding to a fingerprint marked as being in a state to be repaired that does not meet the data integrity condition, the data backup mode is switched to a full backup mode; If the data slices corresponding to the fingerprints marked as being in a state to be repaired all meet the data integrity condition, the data backup mode is switched to the first backup mode; wherein, in the first backup mode, incremental backups of a preset number of backup times are performed; During the data backup process, backup processing is performed on the target data according to the data backup mode.

2. The method according to claim 1, characterized in that When the data backup mode is the first backup mode, performing backup processing on the target data according to the data backup mode includes: Determine whether this is the first backup process for the target data; If not, determine the number of incremental backups that have been performed so far; When it is determined that the number of times the incremental backup has been performed is less than the preset number of times of backup, incremental backup processing is performed on the target data.

3. The method according to claim 2, characterized in that The method further comprises: When it is determined that the number of incremental backups performed is not less than the preset number of backups, performing a partial full backup process on the target data; The process of the partial full backup processing includes: Performing slicing processing on the target data to obtain a plurality of first data slices, and generating a first fingerprint for each of the first data slices; Compare each of the first fingerprints with each of the second fingerprints available in the fingerprint library, determine a target first fingerprint that does not exist in the fingerprint library, and use the target first fingerprint as a newly added fingerprint; The newly added fingerprint and a first data slice corresponding to the first newly added fingerprint are stored, and a first snapshot of the target data in the current backup phase is generated.

4. The method according to claim 2, characterized in that: The performing incremental backup processing on the target data includes: Generate a temporary snapshot of the target data, and obtain a historical snapshot generated when the target data was backed up last time; Compare the temporary snapshot with the historical snapshot to obtain a data change path; Determining new data in the target data according to the data change path; Performing data slicing processing on the newly added data to obtain a plurality of second data slices, and generating a second fingerprint of each of the second data slices; The second data slice and the second fingerprint are stored.

5. The method according to claim 4, characterized in that The method further comprises: A second snapshot is generated according to the historical snapshot and the metadata information of the newly added data, and the second snapshot is stored.

6. The method according to claim 1, characterized in that The method further comprises: Determine whether the data slices corresponding to each fingerprint to be deleted are referenced; If so, the fingerprint corresponding to the referenced data slice is marked as the to-be-repaired state.

7. A data backup device, characterized in that: The device comprises: A detection module, used for detecting whether the data slice corresponding to the fingerprint marked as being in a state to be repaired meets the data integrity condition before executing the data deletion process; A first switching module, configured to switch the data backup mode to a full backup mode if there is at least one data slice corresponding to a fingerprint marked as being in a to-be-repaired state that does not meet the data integrity condition; A second switching module is used to switch the data backup mode to the first backup mode if the data slices corresponding to the fingerprints marked as being in a state to be repaired all meet the data integrity condition; wherein, in the first backup mode, incremental backup of a preset number of backup times is performed; The execution module is used to perform backup processing on the target data according to the data backup mode during the data backup process.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images