A method and system for fast file exclusion for NTFS file system

Abstract

The present invention discloses a method and system for fast file exclusion for NTFS file system, the method comprising: loading a file deletion list; initializing a target volume metafile $MFT; parsing file records in $MFT one by one to generate a file record list; deleting file records corresponding to files or folders with deletion marks in $MFT by comparing the record items in the file record list; deleting the Index information of the deleted files or folders in the Index data of $ObjId and $Reparse files, rearranging the Index data and saving it; updating $Bitmap of $MFT; updating $Bitmap of the target volume. By establishing a file record list to record all files or folders, and marking the files or folders to be deleted with deletion marks, the deletion efficiency is improved and the time for backing up the disk is shortened.

Description

Technical Field

[0001] The invention belongs to the technical field of disk backup, and in particular relates to a method and system for quickly excluding files under an NTFS file system. Background Art

[0002] NTFS stands for New Technology File System in English and is a file system for Windows NT environment. NTFS is a restricted file system for Windows NT family (such as Windows 2000, Windows XP, Windows Vista, Windows 7 and Windows 8.1). NTFS provides long file names, data protection and recovery, and implements security through directory and file permissions. NTFS supports large hard disks and storing files on multiple hard disks. NTFS provides built-in security features that control file affiliation and access. Files on NTFS partitions cannot be directly accessed from DOS or other operating systems.

[0003] The general process of deleting files in the NTFS file system is as follows:

[0004] 1. Find the Index Record (IR) corresponding to the file in the parent directory and get the File record number (FRN) of the file.

[0005] 2. Delete the IR, adjust the order of the Index Record, and save the Index record.

[0006] 3. Release the File Record occupied by the file (mark it as unused) and save the File Record.

[0007] 4. Release the control occupied by the file, set the corresponding data bit of $Bitmap according to the dataruns of the file's properties, and save $Bitmap.

[0008] 5. Adjust the metafiles that record file information such as $ObjId. Items saved by $ObjId are also saved in B* tree sorting. If deleted, they must be re-sorted and saved.

[0009] When backing up the disk, you need to exclude the files or folders that do not need to be backed up. The Windows system API must follow the above general process to delete files or folders. If you delete one file or folder, you must execute the above general process again. If there are many files or folders to be deleted, it will consume a lot of time and system resources, which will increase the time of the entire backup. Summary of the invention

[0010] In view of this, the present invention provides a method and system for fast file exclusion for the NTFS file system, which deletes the files or folders that need to be deleted before backup, without going through the deletion process for each file or folder to be deleted, thereby reducing the time for file deletion.

[0011] In order to solve the above technical problems, the present invention provides a method for quickly eliminating files in the NTFS file system, comprising:

[0012] Load the file deletion list, and open and lock the shadow copy of the target volume to be backed up in read-write mode;

[0013] Initialize the target volume metafile $MFT, obtain the data block list $MFT-DataRuns of $Data of $MFT, and obtain $Bitmap of $MFT;

[0014] Parse the file records in $MFT one by one to form a file record list, the number of record items in the file record list is consistent with the number of file records; record the basic information of the file or folder in the file record list, the basic information includes the deletion mark of the file, the file record number of its parent directory, the deletion mark of the folder, the child deletion mark, the file record number of its parent directory, and the name of the folder;

[0015] Compare the record items in the file record list and delete the file records corresponding to the files or folders with the deletion mark in $MFT; for the record items with the sub-item deletion mark, load the Index data of this folder, delete the Index items corresponding to the sub-items with the deletion mark, and rearrange the non-deleted sub-items and save the index data;

[0016] In the Index data of the $ObjId and $Reparse files, delete the Index information of the deleted files or folders, rearrange the Index data and save it;

[0017] Update $Bitmap of $MFT;

[0018] Update the $Bitmap of the target volume.

[0019] As an improvement, the file records in $MFT are parsed one by one to form a file record list, and the basic information of the file or folder is filled in the file record list, including:

[0020] Refer to the file record header of the used file record in $MFT and generate a file record list, the number of record items in the file record list is consistent with the number of used file records;

[0021] Parse all 0X30 attributes in the used file record to obtain the name of the file or folder and the name of its parent directory, record the file record number of the parent directory of the file into the corresponding record item, and record the name of the folder and the file record number of its parent directory into the corresponding record item;

[0022] The full path of the file or folder is composed of the name of the file or the file and the name of its parent directory, and the full path is compared with the file deletion list to determine whether the file or folder needs to be deleted; a deletion flag is set in the record item corresponding to the file record list for the file or folder to be deleted, and a sub-item deletion flag is set in the parent directory record item corresponding to the file or folder to be deleted.

[0023] As an improvement, $Bitmap in $MFT is used to determine whether the file record is used.

[0024] As a preferred embodiment, if a file or folder occupies two or more file records, its basic information is filled in the record item corresponding to its base file record number in the file record list.

[0025] As an improvement, if a file has multiple 0X30 attributes, assemble the full path for each file name in 0x30, compare it with the file deletion list, and establish an exclusion list for the 0X30 attributes that need to be deleted.

[0026] As an improvement, when parsing all 0X30 attributes in the used file records, if the file record of the parent directory of the file or folder pointed to in the 0X30 attribute of the file or folder has not been analyzed, jump to the file record corresponding to the parent directory for parsing.

[0027] As an improvement, when deleting a file or folder with a deletion mark, if all 0X30 attributes of the file or folder need to be deleted, all file records related to the file or folder are deleted, and the file records and storage space occupied by the corresponding file records are released; if only some 0X30 attributes of the file or folder need to be deleted, the 0X30 attributes that need to be deleted of the file or folder are deleted.

[0028] As an improvement, the non-deleted children are rearranged according to the B*Tree algorithm.

[0029] The present invention also provides a system for quickly eliminating files for the NTFS file system, comprising:

[0030] A file record list creation module, which is used to create a file record list by parsing the file records in $MFT, and record the basic information of the file or folder in the file record list; the basic information includes the deletion mark of the file, the file record number of its parent directory, the deletion mark of the folder, the sub-item deletion mark, the file record number of its parent directory, and the name of the folder;

[0031] The deletion module is used to traverse the file record list, delete the files or folders with the deletion mark, and modify the volume bitmap to release the space occupied by the files or folders;

[0032] The Index data management module is used to adjust the Index of file records with sub-item deletion marks, delete all Index information related to sub-items with deletion marks, and rearrange Index data.

[0033] As an improvement, it also includes:

[0034] $Bitmap update module, used to update $MFT and $Bitmap of target volume.

[0035] The benefit of the present invention lies in that: the method for quickly excluding files for the NTFS file system with the above steps and the system with the above structure record all files or folders by establishing a file record list, and mark the files or folders to be deleted with deletion marks, so as to perform a unified deletion operation, without having to go through the deletion process separately for each file or folder to be deleted, thereby improving the deletion efficiency and shortening the time for backing up the disk. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the present invention.

[0037] Figure 2 It is a structural principle diagram of the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods.

[0039] When backing up a disk, you must first exclude files or folders that do not need to be backed up to save disk space and backup time. The so-called "exclusion" means deleting files and folders that do not need to be backed up from the shadow copy.

[0040] In the NTFS file system, any data in the disk volume exists in the form of "files" or "folders" and is stored in a document called $MFT (Master File Table). The $MFT is composed of several file record arrays. The size of a file record is generally fixed, usually 1KB. A file or folder may occupy multiple file records.

[0041] Each file record contains various attributes of the file. Attributes placed in the file record are called resident attributes, such as 0X30 attributes. If the file content is too large, it can be placed outside the file record (non-resident attributes), and additional clusters are allocated to store it. The used clusters form dataRuns (data flow) information and are saved in the file record.

[0042] like Figure 1 As shown, the present invention provides a method for quickly excluding files for an NTFS file system, comprising:

[0043] S1 loads the file deletion list, and opens and locks the shadow copy of the target volume to be backed up in read-write mode;

[0044] S2 initializes the target volume metafile $MFT, obtains the data block list $MFT-DataRuns of $Data of $MFT, and obtains the $Bitmap corresponding to $MFT;

[0045] S3 parses the file records in $MFT one by one to form a file record list; records basic information of the file or folder in the file record list, wherein the basic information includes a deletion mark of the file, a file record number of its parent directory, a deletion mark of the folder, a sub-item deletion mark, a file record number of its parent directory, and a name of the folder;

[0046] S4 compares the record items in the file record list and deletes the file records corresponding to the files or folders with the deletion mark in $MFT; for the record items with the sub-item deletion mark, load the Index data of this folder, delete the Index items corresponding to the sub-items with the deletion mark, and rearrange the non-deleted sub-items and save the index data;

[0047] S5 deletes the Index information of the deleted file or folder in the Index data of the $ObjId, $Reparse file, rearranges the Index data and saves it;

[0048] S6 updates $Bitmap of $MFT;

[0049] S7 updates the $Bitmap of the target volume.

[0050] The above steps are described in detail below.

[0051] Specifically, in step S1, the shadow copy of the target volume to be backed up (hereinafter referred to as the target volume) is opened and locked in a read-write mode. The so-called locking means that the target volume to be backed up is exclusively used to prevent other programs from performing read and write operations on the target volume to avoid errors.

[0052] After the user selects all the files or folders to be deleted, a file deletion list will be formed, which is loaded when the system starts.

[0053] Specifically, in step S2, the target volume metafile $MFT is initialized, the data block list $MFT-DataRuns of $Data of $MFT is obtained, and the $Bitmap in $MFT is obtained; the data block list $MFT-DataRuns is a list of locations where the metafile $MFT is stored on the disk, and is used to store non-resident attributes of files or folders. The $Bitmap metafile is used to manage the usage of clusters on the volume. Its data stream consists of a series of binary "bits".

[0054] Step S3 specifically includes:

[0055] S31 refers to the file record header of the used file record in $MFT and generates a file record list, wherein the number of record items in the file record list is consistent with the number of file records.

[0056] In the present invention, the basic information of the file or folder to be used includes the deletion flag of the file, the file record number of its parent directory, the deletion flag of the folder, the sub-item deletion flag, the file record number of its parent directory, and the name of the folder. The deletion flag of the file or folder is used to indicate whether the file or folder needs to be deleted, the sub-item deletion flag is used to indicate whether there are sub-items in the folder that need to be deleted, and the name of the folder is used to assemble the full path together with the file name.

[0057] Therefore, it is necessary to establish a file record list AllFileRecord to record the above basic information of files or folders. In this embodiment, the file record list is actually an array, and an element of the array represents a file record. The subscript of the element is the file record number of the file record it represents (this is also the reason why the file record number itself is not recorded, in order to save memory). An element contains several fields for recording the basic information of files or folders.

[0058] A record item in the file record list records the basic properties of a file or folder. The contents of a record item include: 1. Delete flag, whose value is true or false. True means that the file or folder needs to be deleted; 2. For folders, there is a sub-item deletion flag, whose value is true or false; 3. The file record number of the parent directory; 4. If it is a folder, the name of the folder also needs to be recorded.

[0059] In this embodiment, the $Bitmap of the $MFT is first used to determine whether the file record is used. The $Bitmap of the $MFT is a binary stream, which expresses the meaning through "0" and "1". If a "bit" in the $Bitmap of the $MFT is 0, it means that the file record is not used, and the file record is empty and does not need to be paid attention to. If the "bit" is 1, it means that the file record has been used, and a file or folder in the target volume is recorded.

[0060] For the used file records, you need to parse them to get the basic information of the files or folders contained in the file records. Each file record consists of a file record header and several attributes. The file record number of the file or folder is recorded in the file record header, and the name and the file record number of its parent directory are recorded in the attributes.

[0061] In addition, it has been mentioned above that a file or folder may occupy multiple (two or more) file records. For such files or folders, their base file record number is used as the file record number of the file or folder, and their basic information is filled in the record item corresponding to the file record number in the file record list. When a file or folder occupies multiple file records, one of the most important file records is called the base file record, and its record number is called the base file record number. In the present invention, it is only necessary to fill in the basic information of the file or folder in the record item corresponding to its base file record number.

[0062] S32 parses all 0X30 attributes in the used file record to obtain the name of the file or folder and the name of its parent directory, records the file record number of the parent directory of the file into the corresponding record item, and records the name of the folder and the file record number of its parent directory into the corresponding record item;

[0063] The main function of this step is to obtain the name of the file or folder and the name of its parent directory through parsing, and also obtain the file record number of the parent directory, and record the file record number of the parent directory of the file or folder in the record item.

[0064] S33 combines the name of the file or the file with the name of its parent directory to form the full path of the file or folder, and determines whether the file or folder needs to be deleted by comparing the full path with the file deletion list; sets a deletion flag in the record item corresponding to the file record list for the file or folder to be deleted, and sets a sub-item deletion flag in the parent directory record item corresponding to the file or folder to be deleted.

[0065] To determine whether a file or folder is to be deleted, its full path is compared with the file deletion list. Because there may be files or folders with the same name in different paths, the full path is required for accurate judgment.

[0066] If a file has multiple 0X30 attributes, create an exclusion list for the 0X30 attributes that need to be deleted. In the NTFS file system, a file is allowed to have multiple names (only files have hard links, folders do not support this). Users may only want to delete certain file names, so create an exclusion list for the 0X30 attributes of the file that needs to be deleted and add it to the file record list.

[0067] In addition, when parsing all 0X30 attributes in the used file records, if the file record of the parent directory of the file or folder pointed to in the 0X30 attribute of the file or folder has not been analyzed, jump to the file record corresponding to the parent directory for analysis. This is because when $MFT is parsed sequentially, the positions of the file records of the parent-child relationship in $MFT are in a random order. If the child file or folder is in the front, it is necessary to jump to the parent directory to obtain the parent directory information first. This is because when analyzing a file or folder, it is necessary to analyze all its predecessor directories, obtain the full path of the file, and use the full path to determine whether the file or folder has been set to be deleted by the user.

[0068] Specifically, in step S4, the record items in the file record list are compared to find files or folders with deletion marks and some 0X30 attributes of files or folders, and then the corresponding file records in $MFT are deleted.

[0069] When deleting a file or folder with a deletion mark, if all 0X30 attributes of the file or folder need to be deleted, all file records related to the file or folder are deleted, and the data blocks occupied by the corresponding file records are released; if only some 0X30 attributes of the file or folder need to be deleted, the 0X30 attributes that need to be deleted of the file or folder are deleted.

[0070] If a folder has sub-items deleted, load the Index data of this folder, delete the Index item of the sub-item to be deleted, rearrange the undeleted sub-items according to the B*Tree algorithm, and save the index data. The purpose of this step is to update the Index data after deleting files or folders to avoid index errors.

[0071] Specifically, in step S5, in the Index data of the $ObjId, $Reparse file, the Index information of the deleted file or folder is deleted, and the Index data is rearranged and saved. $ObjId, $Reparse is also a metafile inherent in the NTFS file system. After deleting a file or folder, in order to prevent the file system from reporting an error, all related files need to be processed.

[0072] Specifically, in step S6, update the $Bitmap of $MFT

[0073] Specifically, in step S7, $Bitmap of the target volume is updated.

[0074] The present invention also provides a system for quickly eliminating files for the NTFS file system, comprising:

[0075] A file record list creation module is used to create a file record list by parsing the file records in $MFT, and record the basic information of the file or folder in the file record list; the basic information includes the deletion mark of the file, the file record number of its parent directory, the deletion mark of the folder, the sub-item deletion mark, the file record number of its parent directory, and the name of the folder;

[0076] The deletion module is used to compare the record items in the file record list, delete the file records corresponding to the files or folders with deletion marks in $MFT, and modify the volume bitmap to release the space occupied by the files or folders;

[0077] The Index data management module is used to adjust the Index of file records with sub-item deletion marks, delete all Index information related to sub-items with deletion marks, and rearrange Index data.

[0078] $Bitmap update module, used to update $MFT and $Bitmap of target volume.

[0079] By creating a file record list, recording the file record numbers and deletion marks of all files or folders, and then deleting the files or folders to be deleted, there is no need to go through the deletion process for each file or folder to be deleted, which saves system resources and shortens backup time.

[0080] The above are only preferred embodiments of the present invention. It should be noted that the above preferred embodiments should not be regarded as limiting the present invention, and the protection scope of the present invention should be based on the scope defined by the claims. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the spirit and scope of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for fast file exclusion for NTFS file system, characterized in that include: Load the file deletion list, and open and lock the shadow copy of the target volume to be backed up in read-write mode; Initialize the target volume metafile $MFT and obtain the data block list of $Data in $MFT $MFT-DataRuns and get $Bitmap of $MFT; Parse the file records in $MFT one by one, and use $Bitmap of $MFT to determine whether the file record is used; Generate a file record list; Record the basic information of the file or folder in the file record list, the basic information including the deletion mark of the file, the file record number of its parent directory, the deletion mark of the folder, the sub-item deletion mark, the file record number of its parent directory, and the name of the folder; The step of parsing the file records in the $MFT one by one to form a file record list, and filling in the basic information of the file or folder in the file record list includes: consulting the file record headers of the used file records in the $MFT and generating a file record list, wherein the number of record items in the file record list is consistent with the number of the used file records; Parse all 0X30 attributes in the used file record to obtain the name of the file or folder and the name of its parent directory, record the file record number of the parent directory of the file into the corresponding record item, and record the name of the folder and the file record number of its parent directory into the corresponding record item; The full path of the file or folder is formed by combining the name of the file or the file and the name of its parent directory, and the full path is compared with the file deletion list to determine whether the file or folder needs to be deleted; a deletion flag is set in the record item corresponding to the file record list for the file or folder to be deleted, and a sub-item deletion flag is set in the record item of the parent directory corresponding to the file or folder to be deleted; Compare the record items in the file record list and delete the file records corresponding to the files or folders with the deletion mark in $MFT; for the record items with the sub-item deletion mark, load the Index data of this folder, delete the Index items corresponding to the sub-items with the deletion mark, and rearrange the non-deleted sub-items and save the index data; In the Index data of the $ObjId and $Reparse files, delete the Index information of the deleted files or folders, rearrange the Index data and save it; Update $Bitmap of $MFT; Update the $Bitmap of the target volume.

2. The method for fast file exclusion for NTFS file system according to claim 1, characterized in that: If a file or folder occupies two or more file records, its basic information is filled in the record item corresponding to its base file record number in the file record list.

3. The method for fast file exclusion for NTFS file system according to claim 1, characterized in that: If a file has multiple 0X30 attributes, assemble the full path for each file name in 0x30, compare it with the file deletion list, and create an exclusion list for the 0X30 attributes that need to be deleted.

4. The method for fast file exclusion for NTFS file system according to claim 1, characterized in that: When parsing all 0X30 attributes in the used file records, if the file record of the parent directory of the file or folder pointed to in the 0X30 attribute of the file or folder has not been analyzed, jump to the file record corresponding to the parent directory for analysis.

5. The method for fast file exclusion for NTFS file system according to claim 1, characterized in that: When deleting a file or folder with a deletion mark, if all 0X30 attributes of the file or folder need to be deleted, all file records related to the file or folder are deleted, and the file records and storage space occupied by the corresponding file records are released; If only part of the 0X30 attributes of a file or folder need to be deleted, delete the 0X30 attributes of the file or folder that need to be deleted.

6. The method for fast file exclusion for NTFS file system according to claim 1, characterized in that: Rearrange the non-deleted children according to the B*Tree algorithm.

7. A system for fast file exclusion for NTFS file system, characterized in that include: The file record list creation module is used to create a file record list by parsing the file records in $MFT one by one, using $Bitmap of $MFT to determine whether the file record is used, and recording the basic information of the file or folder in the file record list; The basic information includes the deletion mark of the file, the file record number of its parent directory, the deletion mark of the folder, the deletion mark of the sub-item, the file record number of its parent directory, and the name of the folder; The step of parsing the file records in the $MFT one by one to form a file record list, and filling in the basic information of the file or folder in the file record list includes: consulting the file record headers of the used file records in the $MFT and generating a file record list, wherein the number of record items in the file record list is consistent with the number of the used file records; Parse all 0X30 attributes in the used file record to obtain the name of the file or folder and the name of its parent directory, record the file record number of the parent directory of the file into the corresponding record item, and record the name of the folder and the file record number of its parent directory into the corresponding record item; The full path of the file or folder is formed by combining the name of the file or the file and the name of its parent directory, and the full path is compared with the file deletion list to determine whether the file or folder needs to be deleted; a deletion flag is set in the record item corresponding to the file record list for the file or folder to be deleted, and a sub-item deletion flag is set in the record item of the parent directory corresponding to the file or folder to be deleted; The deletion module is used to compare the record items in the file record list, delete the file records corresponding to the files or folders with deletion marks in $MFT, and modify the volume bitmap to release the space occupied by the files or folders; Index data management module, used to adjust the index of file records with sub-item deletion marks, delete all index information related to sub-items with deletion marks, and rearrange index data; $Bitmap update module, used to update $MFT and $Bitmap of target volume.

Images