Directory file content increment snapshot backup method

Abstract

The invention relates to the technical field of computers and network storage, in particular to a directory file content incremental snapshot backup method, which comprises the following steps of: S1, judging whether a reference directory exists or not, if not, executing a step S2, otherwise, executing a step S3; s2, creating a reference directory; s3, generating reference directory file consistency verification information according to the reference directory file, wherein the reference directory file consistency verification information is used for judging whether file contents have differences or not; s4, searching for a difference file which is modified in the time period and is inconsistent with the content of the consistency verification information of the directory file; s5, comparing the difference file with the reference directory file to generate a snapshot backup containing difference data; s6, snapshot backup consistency information is calculated and written into the snapshot file; and S7, merging the difference data in the snapshot backup into the reference directory to form a new reference directory. According to the method, the minimum-granularity snapshot backup is carried out on the Linux system through line-level addition, deletion and modification of file content, the snapshot backup only records data changes, snapshot consistency verification and compression are supported, and storage space occupation is reduced.

Description

Technical Field

[0001] The invention relates to the technical field of computer and network storage, and in particular to a directory file content incremental snapshot backup method. Background Art

[0002] Incremental differential backup of directories on Linux systems often relies on rsync tools, Btrfs file system, and ZFS file system, but they have the following defects: the rsync tool cannot support small-granular differential backup at the file content level and can only back up the entire file; in addition, the use of Btrfs file system or ZFS file system requires special file system support, which limits the flexibility of the backup solution and reduces the file read and write performance in the system. As the number of snapshot chains increases, the performance impact becomes greater. In addition, the special file system solution relies on binary data as the differential storage unit, so the specific data changes cannot be directly seen from the snapshot file. Summary of the invention

[0003] The purpose of the present invention is to provide a method for incremental snapshot backup of directory file content, in which the minimum difference unit recorded is the content line, and the incremental difference backup at multiple time points can be realized in the Linux system without relying on a special file system.

[0004] To achieve the above object, the present invention provides the following technical solution: a method for incremental snapshot backup of directory file content, comprising: S1. Determine whether the reference directory exists. If not, proceed to step S2; otherwise, proceed to step S3; S2. Create a reference directory; S3. Generates reference directory file consistency check information based on the reference directory file, used to determine whether there are differences in the file content; S4. Find the difference files that have been modified within the time period and are inconsistent with the directory file consistency check information; S5. Compare the difference file with the reference directory file to generate a snapshot backup containing the difference data; S6. Calculate the snapshot backup consistency information and write it into the snapshot file; S7. Merge the differential data in the snapshot backup into the reference directory to form a new reference directory.

[0005] Preferably, the method further includes S8 of performing consistency verification again on the generated reference directory file.

[0006] Preferably, S9 also includes using a tar archiving tool in combination with an xz compression algorithm to compress the snapshot backup file.

[0007] Preferably, the consistency check information of the reference directory file is performed through MD5.

[0008] Preferably, step S4 uses the find command to perform a specified search to speed up the search for the modified files at the specified time. Preferably, step S5 compares the difference file with the reference file through a diff command to generate a snapshot backup containing the difference data.

[0009] Preferably, the step S7 merges the difference data into the reference directory through a patch command.

[0010] Compared with the prior art, the invention has the following beneficial effects: the invention realizes small-grained snapshot backup of overall file changes (deletion and addition of the entire file) and file content changes (row-level addition, deletion and modification) in the Linux native file system. This snapshot backup only records the changed data, supports snapshot consistency verification and compression, and reduces storage space usage. In addition, the method supports the generation of snapshot backups at multiple time points, and users can restore directory data to any specified snapshot time point data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of specific implementation steps of the present invention; DETAILED DESCRIPTION The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] Example: In this embodiment, a directory file content incremental snapshot backup method includes: S1. Determine whether the reference directory exists. If not, proceed to step S2; otherwise, proceed to step S3; S2. Create a reference directory; S3. Generate MD5 consistency check information of the reference directory file based on the reference directory file to determine whether there is a difference in the file content; S4. Use the find command to specify the difference files that have been modified within the specified time period and are inconsistent with the directory file consistency check information. The find command speeds up the search for the modified files within the specified time period. The found files can only indicate that the files have been modified within the time period. They may have been modified multiple times, but the contents are not necessarily inconsistent. Therefore, MD5 verification is required to check whether the contents are consistent and filter out misjudged items. S5. Use the diff command to compare the difference file with the reference directory file to generate a snapshot backup containing the difference data, find out the inconsistent information of the file content line data, record the minimum difference unit as the content line, not the data block, and record the line number information and context flag as well as the difference data. If it is found that the file is deleted or added, record it as a file unit; S6. Calculate the snapshot backup consistency information and write it into the snapshot file to ensure data consistency during snapshot backup and restore, and prevent file loss and tampering; S7. Use the patch command to merge the difference data in the snapshot backup into the reference directory to form a new reference directory. Keep the reference directory containing the differences of all snapshots to ensure that the difference of the next snapshot comparison is the difference data not included in the previous snapshot.

[0013] Optionally, S8 may be further included to perform consistency verification on the generated reference directory file again to further verify whether the file content is actually modified, thereby reducing the files for difference comparison and shortening the difference snapshot backup time.

[0014] Optionally, S9 uses the tar archiving tool combined with the xz compression algorithm to compress the snapshot backup file, which can greatly compress the snapshot volume. At the same time, the snapshot can be partially decompressed for restoration as needed, without the need to decompress all of it, saving decompression time.

[0015] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for incremental snapshot backup of directory file contents, the other features of which are: S1. Determine whether the reference directory exists. If not, proceed to step S2; otherwise, proceed to step S3; S2. Create a reference directory; S3. Generates reference directory file consistency check information based on the reference directory file, used to determine whether there are differences in the file content; S4. Find the difference files that have been modified within the time period and are inconsistent with the directory file consistency check information; S5. Compare the difference file with the reference directory file to generate a snapshot backup containing the difference data; S6. Calculate the snapshot backup consistency information and write it into the snapshot file; S7. Merge the differential data in the snapshot backup into the reference directory to form a new reference directory.

2. The method for incremental snapshot backup of directory file contents according to claim 1, characterized in that: It also includes S8 re-checking the consistency information of the generated reference directory file.

3. The method for incremental snapshot backup of directory file contents according to claim 2, characterized in that: It also includes S9 using the tar archiving tool combined with the xz compression algorithm to compress snapshot backup files.

4. The method for incremental snapshot backup of directory file contents according to claim 1 or 2, characterized in that: Use MD5 to verify the consistency of the reference directory file.

5. The method for incremental snapshot backup of directory file contents according to claim 1, characterized in that: The step S4 performs a specified search through the find command to speed up the search for the modified files at the specified time.

6. The method for incremental snapshot backup of directory file contents according to claim 1, characterized in that: The step S5 compares the difference file with the reference file by using the diff command to generate a snapshot backup containing the difference data.

7. The method for incremental snapshot backup of directory file contents according to claim 1, characterized in that: The step S7 merges the difference data into the reference directory through the patch command.

Images