A method and system for detecting changes in VSS and NTFS hot backup blocks

By reading the NTFS volume allocation bitmap and combining VSS snapshots with CRC snapshot verification, the problems of unstable data block status and low efficiency of incremental backup in hot backup are solved, and data consistency and continuity of backup process are achieved.

CN122653890BActive Publication Date: 2026-11-17BEIJING ANDY TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202611114976.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-07-27
Publication Date
2026-11-17
Estimated Expiration
2046-07-27

AI Technical Summary

Technical Problem

In hot backup scenarios, existing technologies suffer from problems such as unstable data block status baseline, large scanning range for incremental backup change detection, misjudgment due to metadata changes, and backup interruption in abnormal scenarios.

Method used

The allocated cluster set is determined by reading the volume allocation bitmap of the NTFS volume, a VSS snapshot is created as a consistent read point, and a two-level detection mechanism is constructed by using MFT metadata comparison and CRC snapshot verification to narrow the scope of change detection and eliminate metadata misjudgment. In abnormal scenarios, a hierarchical degradation process is set up.

Benefits of technology

Ensure backup data consistency, significantly shorten backup time, accurately identify data block changes, ensure backup process continuity, and avoid misjudgments and backup interruptions caused by metadata changes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122653890B_ABST
    Figure CN122653890B_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for detecting changes in VSS and NTFS hot backup blocks, belonging to the field of computer data backup technology. This solution aggregates allocated clusters into block-level baselines using a volume allocation bitmap and creates VSS snapshots to generate CRC and MFT parent state snapshots. During incremental backups, candidate changed files are first screened by comparing the MFT snapshot and converted into LBA sets. Then, the actual changed blocks are confirmed through CRC verification, and deleted blocks are identified using the difference between the parent CRC snapshot and the bitmap. When metadata is abnormal, the scan is automatically expanded while retaining CRC confirmation, forming a hierarchical degradation mechanism that balances consistency and backup continuity, significantly reducing incremental detection overhead.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of computer data backup technology, specifically relating to a method and system for detecting changes in VSS and NTFS hot backup blocks. Background Technology

[0002] As enterprises deepen their IT infrastructure development, the amount of data generated by various business systems continues to grow. Critical information such as database files, business logs, configuration files, and user data are stored on disks. In the event of physical disk damage, file system logical errors, human error, or ransomware attacks, irrecoverable data loss may occur, leading to business interruption and severe economic losses. Therefore, data backup has become a fundamental means of ensuring business continuity and data security.

[0003] Traditional data backup methods mainly include manual copying and planned downtime backups. Manual copying relies on maintenance personnel to periodically copy data to external storage media, which is inefficient and prone to omissions. Downtime backups require suspending the operation of business systems during the backup window, which is increasingly unacceptable in scenarios with extremely high availability requirements, such as financial trading systems and online service platforms. As business systems evolve towards 24 / 7 uninterrupted operation, the need for hot backups to perform data backups during continuous operation of business systems is becoming increasingly urgent.

[0004] As disk capacity grows from hundreds of GB to several TB or even tens of TB, full backups require copying all data from the entire disk volume block by block to backup storage. Full backups not only consume significant time, network bandwidth, and backup storage space, but the continuous read operations on the source disk within the backup window also impose additional I / O load on running business systems. To reduce backup overhead, incremental backup technology has been widely adopted. Its core idea is that after the initial full backup, subsequent backups only process data that has changed since the last backup.

[0005] Existing incremental backup technologies typically determine data changes from two dimensions: First, based on changes in file metadata, such as comparing file modification timestamps, file sizes, and other attributes to determine if a file has been modified. A typical implementation is comparing file record attributes in the Master File Table (MFT). However, modifications to file attributes (such as only changing file permissions or extended attributes), moving files between directories, and adjusting the directory index structure all lead to changes in metadata, but the actual data content of the file remains unchanged. If a file needs to be backed up based solely on metadata changes, a large amount of unchanged data will be repeatedly backed up, resulting in a waste of backup space and transmission bandwidth. Second, based on changes in data block content, such as dividing the disk volume into data blocks of a fixed size, calculating digital fingerprints (such as MD5 values ​​or CRC checksums) for each block, and comparing them with the fingerprints of corresponding blocks in historical backups. This method can accurately determine whether the data block content has changed, but in large-capacity disk scenarios, the overhead of scanning and fingerprinting all data blocks of the entire volume is extremely high, severely impacting the execution efficiency of incremental backups.

[0006] Furthermore, in hot backup scenarios, the business system continuously writes to the source volume during the backup process. If the backup program reads data directly from the source volume, data blocks read at different times may belong to different write points, making it impossible to guarantee the consistency of the backup data. Although the Windows operating system provides Volume Shadow Copy Service (VSS) to create a consistent snapshot of the source volume at a certain point in time, using VSS alone can only solve the read consistency problem and cannot solve the problem of low efficiency in detecting incremental changes on large-capacity disks.

[0007] In the prior art, patent document CN104375905A discloses a method and system for incremental backup based on data blocks. It determines whether the content of a data block has changed by verifying its digital fingerprint (MD5 value), and only backs up data blocks with different fingerprints. However, this method requires pre-dividing all data into fixed-size data blocks and calculating a fingerprint for each block. When dealing with large-capacity disks, the cost of full fingerprint calculation and block-by-block comparison is too high, and the method does not address how to utilize file system metadata to narrow the scope of change detection. Patent document CN111414280A discloses a data backup system and method for the NTFS file system. It determines the validity of data blocks by reading the NTFS volume allocation bitmap ($Bitmap), copying only valid data blocks, thereby skipping unallocated space to improve the speed of full backup. However, this method is only applicable to full backup scenarios and does not address how to detect data block changes in incremental backups. Patent document CN119883741A discloses a method to improve the speed of NTFS incremental backups. It identifies changed files by comparing the current MFT record on the disk with historical MFT records saved in the image, and uses an exclusion block algorithm to mark data blocks that do not need backup. However, this method still relies on changes in MFT metadata as the basis for judgment, and cannot distinguish between metadata changes and data content changes. This may lead to files whose content has remained unchanged but whose metadata has changed being incorrectly marked as needing backup. Furthermore, this method does not provide a backup guarantee mechanism for abnormal scenarios such as MFT read failure or bitmap acquisition failure.

[0008] In summary, the following technical issues urgently need to be addressed in current hot backup scenarios: First, how to establish a stable data block state baseline while the business system is continuously writing data to ensure the consistency of backup data; second, how to reduce the change detection scan range during incremental backups of large-capacity disks and reduce block-by-block scan overhead; third, how to eliminate misjudgments caused by changes in metadata rather than data content while utilizing file system metadata to narrow the change detection range; and fourth, how to continue the backup process through degradation processing in abnormal scenarios where some metadata or system interfaces are unavailable, rather than directly interrupting the backup task. Summary of the Invention

[0009] To address the shortcomings of the existing technologies, this application provides a method and system for detecting changes in VSS and NTFS hot backup blocks, aiming to solve the following technical problems existing in the current hot backup technology: how to establish a stable data block state baseline to ensure backup data consistency under continuous writing conditions of the business system; how to reduce the change detection scan range during incremental backup of large-capacity disks; how to eliminate misjudgments caused by metadata changes while using file system metadata to narrow the detection range; and how to continue the backup process in abnormal scenarios where some metadata or system interfaces are unavailable.

[0010] In its first aspect, this application proposes a method for detecting changes in VSS and NTFS hot backup blocks, including the following steps:

[0011] Read the volume allocation bitmap of the NTFS volume to be backed up, determine the allocated cluster set, and aggregate the allocated cluster set into a block-level LBA baseline according to the preset backup block size;

[0012] Create a Volume Shadow Copy Service (VSS) snapshot of the NTFS volume to be backed up as a consistent read point. Read data blocks from the VSS snapshot according to the block-level LBA baseline, write the read data blocks to the backup image, and generate block index, cyclic redundancy check snapshot and master file table snapshot as parent status information.

[0013] During subsequent backups, the master file table snapshot is compared with the master file table information in the current VSS snapshot to filter candidate changed files, and the data run list of the candidate changed files is converted into a candidate changed LBA set;

[0014] For LBAs in the candidate change LBA set, read the corresponding data block from the current VSS snapshot and calculate the check value. Compare the calculated check value with the corresponding LBA check value of the cyclic redundancy check snapshot in the parent status information. LBAs with different check values ​​are determined to be real change blocks.

[0015] The block-level LBA set aligned with the current volume allocation bitmap is used as the current allocated block set, and the LBA recorded in the cyclic redundancy check snapshot in the parent status information is used as the parent status block set. The block to be deleted is determined based on the parent status block set and the current allocated block set.

[0016] Further, the step of reading the volume allocation bitmap of the NTFS volume to be backed up includes:

[0017] The volume allocation bitmap is read first through the master file table path: the meta file record corresponding to the volume allocation bitmap is located according to the starting position of the master file table, the data attributes and data run list in the meta file record are parsed, and the volume allocation bitmap content is read according to the cluster range indicated by the data run list to obtain the allocated cluster set;

[0018] If reading the main file table path fails, the system volume allocation bitmap query interface is called to obtain the volume allocation status.

[0019] If the system volume allocation bitmap query interface call fails, but the total volume size can be obtained, then the backup block range is formed based on the entire volume range.

[0020] Furthermore, the step of preferentially reading the volume allocation bitmap via the master file table path includes:

[0021] Read the boot sector of the NTFS volume to be backed up and obtain the starting cluster number of the master file table;

[0022] The file record in the master file table is located based on the starting cluster number of the master file table, and the record number of the file record is 6;

[0023] The data attributes of the file record are parsed. If the data attribute is a resident attribute, the volume allocation bitmap content is read directly from the attribute header. If the data attribute is a non-resident attribute, the data run list is parsed, and the volume allocation bitmap content is read from the volume according to the starting cluster number and the number of clusters in the data run list.

[0024] Further, the aggregation of the allocated cluster set into a block-level LBA baseline according to a preset backup block size includes:

[0025] The logical space of the volume to be backed up is divided into contiguous backup blocks according to the preset backup block size, with each backup block covering multiple clusters;

[0026] For each backup block, check whether there is at least one cluster within its covered cluster range that belongs to the allocated cluster set;

[0027] If it exists, the LBA of the backup block will be included in the block-level LBA baseline.

[0028] Further, the comparison of the master file table snapshot with the master file table information in the current VSS snapshot to filter candidate changed files includes:

[0029] Using the file reference number as an index, for file records with the same file reference number in the main file table snapshot and the current main file table information, compare the file record's usage status flag, file modification time, file data size, and data execution list summary;

[0030] If the usage status flag of a file record changes from "in use" to "unused", it is determined to be a candidate file for deletion; if a file reference number exists in the current main file table information but does not exist in the main file table snapshot, it is determined to be a candidate file for addition.

[0031] If a file is modified later than the modification time recorded in the master file table snapshot, the file data size is different from that recorded in the master file table snapshot, or the data run list summary is different from that recorded in the master file table snapshot, then it is determined to be a candidate file that may have been modified.

[0032] Further, the LBAs in the candidate change LBA set are compared using check values, including:

[0033] Read the data content of the backup block corresponding to the candidate change LBA from the current VSS snapshot, and calculate the check value using the same cyclic redundancy check algorithm as when generating the cyclic redundancy check snapshot;

[0034] Read the historical parity value of the corresponding LBA from the Cyclic Redundancy Check snapshot in the parent status information;

[0035] If the currently calculated checksum is different from the historical checksum, the data block corresponding to the LBA is determined to be a real changed block and written to the current backup image; if they are the same, the data block is determined to have no content change and is not written to the current backup image.

[0036] Further, the step of determining the block to be deleted based on the parent state block set and the currently allocated block set includes:

[0037] Perform a difference operation on the parent state block set and the current allocated block set, and determine the LBA that exists in the parent state block set but does not exist in the current allocated block set as a deleted block;

[0038] Record the LBA and deletion marker of the deleted block in the current backup image.

[0039] Furthermore, the creation of a Volume Shadow Copy Service (VSS) snapshot of the NTFS volume to be backed up, serving as a consistent read point, and reading data blocks from the VSS snapshot based on block-level LBA benchmarks, includes:

[0040] Initiate a snapshot creation request to the volume shadow copy service to generate a consistent data view of the NTFS volume to be backed up at the time of snapshot creation;

[0041] When the business system continuously writes to the source volume, the data content at the time of snapshot creation is read through the write-on-write replication mechanism of VSS snapshot, so that all data blocks read during the backup process belong to the same point in time.

[0042] Furthermore, it also includes a tiered degradation processing step:

[0043] When the master file table snapshot in the parent status information does not exist, the current master file table information reading fails, the data execution list of the candidate file cannot be completely parsed, or a file record that cannot be conservatively mapped (i.e., a file record whose data block position cannot be determined by the regular mapping rules) is detected, the step of comparing the master file table snapshot with the current master file table information is skipped. All allocated blocks in the current block-level LBA baseline are taken as candidate change ranges, and check values ​​are calculated block by block and compared with the cyclic redundancy check snapshot in the parent status information to confirm the real changed blocks.

[0044] When reading the volume allocation bitmap from the master file table path fails, the system volume allocation bitmap query interface is called to obtain the volume allocation status.

[0045] When the system volume allocation bitmap query interface fails to call but the total volume size can be obtained, a backup block range is formed based on the entire volume range.

[0046] Secondly, this application proposes a VSS and NTFS hot backup block change detection system, including:

[0047] The block status baseline construction module is used to read the volume allocation bitmap of the NTFS volume to be backed up, determine the allocated cluster set, and aggregate the allocated cluster set into a block-level LBA baseline according to the preset backup block size.

[0048] The VSS snapshot read module is used to create a VSS snapshot of the NTFS volume to be backed up as a consistent read point, read data blocks from the VSS snapshot according to the block-level LBA baseline, and write the read data blocks to the backup image.

[0049] The parent state generation module is used to generate block indexes, cyclic redundancy check snapshots, and master file table snapshots as parent state information;

[0050] The candidate change location module is used to compare the master file table snapshot with the master file table information in the current VSS snapshot during subsequent backups, filter candidate change files, and convert the data run list of candidate change files into a candidate change LBA set.

[0051] The change confirmation module is used to read the corresponding data block from the current VSS snapshot and calculate the check value for LBA in the candidate change LBA set. The calculated check value is compared with the corresponding LBA check value in the cyclic redundancy check snapshot in the parent status information. LBA with different check values ​​are determined to be real change blocks.

[0052] The deletion block identification module is used to take the block-level LBA set after the current volume allocation bitmap is aligned as the current allocated block set, take the LBA recorded in the cyclic redundancy check snapshot in the parent status information as the parent status block set, and determine the deletion block based on the parent status block set and the current allocated block set.

[0053] The tiered degradation processing module is used to perform tiered degradation processing when anomalies occur in the master file table snapshot path, volume allocation bitmap read path, or data run list parsing path, in order to maintain the continuity of the backup process by expanding the scan scope.

[0054] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0055] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0056] The beneficial effects of this invention are:

[0057] This solution establishes a block-level state baseline based on the volume allocation bitmap during hot backup and uses VSS snapshots as consistent data read points. It leverages a copy-on-write mechanism to ensure that all data blocks read during full and incremental backups belong to the same point in time. A two-level detection mechanism consisting of MFT candidate location and CRC snapshot verification is employed. First, MFT metadata comparison reduces the change detection scope from the entire volume to a set of candidate change LBAs. Then, CRC verification is performed only on candidate LBAs to eliminate false positives for metadata changes. This reduces the scale of block-by-block verification required for incremental backups from the entire volume level to the candidate change level, significantly shortening the backup time window. Simultaneously, the parent CRC... The LBA set of the Snapshot is compared with the block-level LBA set aligned with the current volume allocation bitmap to accurately identify released deleted blocks and record deletion marks, ensuring the accuracy of the backup status in the incremental backup chain. In addition, a multi-level degradation processing path is set up. When the MFT path, volume allocation bitmap reading path, or data run list parsing path is abnormal, it automatically switches to a degradation mode with an expanded scan range without interrupting the backup. After degradation, the ability to confirm changes in CRCSnapshot and generate parent status is still maintained, and subsequent incremental backups can continue. Finally, after each round of backup is completed, a new complete CRC Snapshot is generated based on the parent CRC Snapshot. It inherits the checksums of unchanged blocks, overwrites the checksums of changed blocks, and removes deleted block entries, providing a continuous parent status basis for subsequent incremental backups and fundamentally avoiding chain breaks. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the overall process of S1-S5 in an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram of the overall process of steps A1-A8 provided in an embodiment of the present invention.

[0060] Figure 3 This is a schematic diagram of the sub-process for constructing a block state baseline in an embodiment of the present invention.

[0061] Figure 4 This is a schematic diagram of a sub-process for candidate change localization based on MFT Snapshot in an embodiment of the present invention.

[0062] Figure 5 This is a system framework diagram in an embodiment of the present invention. Detailed Implementation

[0063] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein; rather, these embodiments are provided so that a more thorough understanding of the invention can be achieved and that the full scope of the invention can be conveyed to those skilled in the art.

[0064] For ease of understanding, the following explanations are provided for NTFS file system proprietary terms used in this specification: Names beginning with "$", such as "$Bitmap", "$DATA", and "$STANDARD_INFORMATION", appearing in this application are standard naming conventions for metafiles or file record attributes in the NTFS file system. Here, "$" is an inherent prefix for NTFS metafiles and attributes, used to distinguish them from ordinary user files or attributes; "Bitmap" refers to the volume allocation bitmap; "DATA" refers to the file data attribute; and "STANDARD_INFORMATION" refers to the standard information attribute. These terms are all standard names commonly used by those skilled in the art in NTFS file system related technologies.

[0065] Firstly, this application proposes a method for detecting changes in VSS and NTFS hot backup blocks, such as... Figure 1 As shown, it includes the following steps:

[0066] S1: Read the volume allocation bitmap of the NTFS volume to be backed up, determine the allocated cluster set, and aggregate the allocated cluster set into a block-level LBA (Logical Block Addressing) base according to the preset backup block size;

[0067] Further, the step of reading the volume allocation bitmap of the NTFS volume to be backed up includes:

[0068] The volume allocation bitmap is read first through the master file table path: the meta file record corresponding to the volume allocation bitmap is located according to the starting position of the master file table, the data attributes and data run list in the meta file record are parsed, and the volume allocation bitmap content is read according to the cluster range indicated by the data run list to obtain the allocated cluster set;

[0069] If reading the main file table path fails, the system volume allocation bitmap query interface is called to obtain the volume allocation status.

[0070] If the system volume allocation bitmap query interface call fails, but the total volume size can be obtained, then the backup block range is formed based on the entire volume range.

[0071] Furthermore, the step of preferentially reading the volume allocation bitmap via the master file table path includes:

[0072] Read the boot sector of the NTFS volume to be backed up and obtain the starting cluster number of the master file table;

[0073] The file record in the master file table is located based on the starting cluster number of the master file table, and the record number of the file record is 6;

[0074] The data attributes of the file record are parsed. If the data attribute is a resident attribute, the volume allocation bitmap content is read directly from the attribute header. If the data attribute is a non-resident attribute, the data run list is parsed, and the volume allocation bitmap content is read from the volume according to the starting cluster number and the number of clusters in the data run list.

[0075] Further, the aggregation of the allocated cluster set into a block-level LBA baseline according to a preset backup block size includes:

[0076] The logical space of the volume to be backed up is divided into contiguous backup blocks according to the preset backup block size, with each backup block covering multiple clusters;

[0077] For each backup block, check whether there is at least one cluster within its covered cluster range that belongs to the allocated cluster set;

[0078] If it exists, the LBA of the backup block will be included in the block-level LBA baseline.

[0079] The backup program reads the file system metadata of the NTFS volume to be backed up, obtaining parameters such as bytes per sector, bytes per cluster, sectors per cluster, MFT starting cluster number, and MFT record size. The program prioritizes reading the $Bitmap via MFT, locating the $Bitmap metadata record based on the MFT starting position, parsing the $DATA attribute and Data Runs within the record, reading the $Bitmap content according to the cluster range indicated by the Data Runs, and obtaining the set of allocated clusters in the volume. If MFT reading fails, the system Bitmap query interface FSCTL_GET_VOLUME_BITMAP is used to obtain the volume allocation status. If the system interface still cannot obtain a valid Bitmap but can obtain the volume size, a backup block range is formed based on the entire disk to ensure the backup process continues. The program aggregates the set of allocated clusters in the $Bitmap into a block-level LBA base according to the preset backup block size. The backup block size is typically larger than the NTFS cluster size, and one backup block covers multiple clusters. When at least one allocated cluster exists within the coverage area of ​​a backup block, that backup block is included in the backup block range. This process converts the cluster-level Bitmap state into a backup block-level LBA baseline, ensuring that subsequent Block Indexes and CRCSnapshots use consistent block coordinates.

[0080] The $Bitmap block status baseline is only used to determine which blocks are currently allocated or need to be included in the detection scope; it cannot determine whether these blocks have undergone content changes relative to the parent backup. Therefore, this method, after establishing the block status baseline, also combines it with a CRC snapshot to confirm changes in block content. In other words, the Bitmap is used to limit the block range, and the CRC snapshot is used to confirm the change results; the two work together to complete block-level change detection in hot backup.

[0081] S2: Create a Volume Shadow Copy Service (VSS) snapshot of the NTFS volume to be backed up as a consistent read point, read data blocks from the VSS snapshot according to the block-level LBA baseline, write the read data blocks to the backup image, and generate block index, cyclic redundancy check snapshot and master file table snapshot as parent status information;

[0082] Furthermore, the creation of a Volume Shadow Copy Service (VSS) snapshot of the NTFS volume to be backed up, serving as a consistent read point, and reading data blocks from the VSS snapshot based on block-level LBA benchmarks, includes:

[0083] Initiate a snapshot creation request to the volume shadow copy service to generate a consistent data view of the NTFS volume to be backed up at the time of snapshot creation;

[0084] When the business system continuously writes to the source volume, the data content at the time of snapshot creation is read through the write-on-write replication mechanism of VSS snapshot, so that all data blocks read during the backup process belong to the same point in time.

[0085] The program creates a VSS snapshot as a consistent read point and reads data from the VSS snapshot according to the block-level LBA baseline. VSS uses a copy-on-write mechanism to save the data state at the time of snapshot creation; when the business system continues to write to the source volume during backup, if a data block is modified after the snapshot is created, VSS retains the original content of that data block at the time of snapshot creation. The backup program reads data from the VSS snapshot, thereby obtaining a data view at the same point in time during the backup process.

[0086] The program writes the read data to the backup image and generates a Block Index, a CRC Snapshot, and an MFT Snapshot. The Block Index records the LBA of the backup block, the data location within the image, and block checksum information; the CRC Snapshot records the CRC checksum of each LBA at the same block granularity; and the MFT Snapshot records file metadata and Data Runs information. All three are stored together as a parent state, with the MFT Snapshot used for subsequent candidate location, the CRC Snapshot for subsequent block-level confirmation, and the Block Index for image data location and recovery.

[0087] S3: During subsequent backups, compare the master file table snapshot with the master file table information in the current VSS snapshot, filter candidate changed files, and convert the data run list of the candidate changed files into a candidate changed LBA set;

[0088] Further, the comparison of the master file table snapshot with the master file table information in the current VSS snapshot to filter candidate changed files includes:

[0089] Using the file reference number as an index, for file records with the same file reference number in the main file table snapshot and the current main file table information, compare the file record's usage status flag, file modification time, file data size, and data execution list summary;

[0090] If the usage status flag of a file record changes from "in use" to "unused", it is determined to be a candidate file for deletion; if a file reference number exists in the current main file table information but does not exist in the main file table snapshot, it is determined to be a candidate file for addition.

[0091] If a file is modified later than the modification time recorded in the master file table snapshot, the file data size is different from that recorded in the master file table snapshot, or the data run list summary is different from that recorded in the master file table snapshot, then it is determined to be a candidate file that may have been modified.

[0092] During subsequent backups, the program reads the CRC Snapshot and MFT Snapshot from the parent image and the MFT information from the current VSS snapshot. The program compares the file references, modification times, data sizes, and Data Run summaries in the parent MFT Snapshot and the current MFT information to filter out candidate files that have been added, modified, or deleted. The MFT comparison results are only used as a candidate range and are not directly used as a conclusion about block changes.

[0093] For candidate files, the program parses the Data Runs of their $DATA attribute, converting the file-level candidate ranges into candidate change LBAs. During the conversion, the cluster ranges in the Data Runs are converted into LBAs and aligned according to the backup block size; LBAs from multiple files or multiple Data Runs are merged and deduplicated to obtain a set of candidate change blocks. Hollow regions in sparse files that have not actually been allocated physical clusters are not included in the set of candidate change blocks.

[0094] S4: For LBAs in the candidate change LBA set, read the corresponding data block from the current VSS snapshot and calculate the check value. Compare the calculated check value with the corresponding LBA check value of the cyclic redundancy check snapshot in the parent status information. LBAs with different check values ​​are determined to be real change blocks.

[0095] Further, the LBAs in the candidate change LBA set are compared using check values, including:

[0096] Read the data content of the backup block corresponding to the candidate change LBA from the current VSS snapshot, and calculate the check value using the same cyclic redundancy check algorithm as when generating the cyclic redundancy check snapshot;

[0097] Read the historical parity value of the corresponding LBA from the Cyclic Redundancy Check snapshot in the parent status information;

[0098] If the currently calculated checksum is different from the historical checksum, the data block corresponding to the LBA is determined to be a real changed block and written to the current backup image; if they are the same, the data block is determined to have no content change and is not written to the current backup image.

[0099] The program reads data from the current VSS snapshot only for candidate change LBAs and calculates their CRC. The calculated result is compared with the checksum of the corresponding LBA in the parent CRC snapshot. If they differ, the block is confirmed as a genuine change block; if they match, it means the MFT candidate change did not cause a change in the data block content, and the block is not written to this backup image. In this way, the MFT narrows the scan scope, and the CRC snapshot eliminates false positives caused by changes in MFT metadata.

[0100] S5: Use the block-level LBA set after aligning the current volume allocation bitmap as the current allocated block set, use the LBA recorded in the cyclic redundancy check snapshot in the parent status information as the parent status block set, and determine the block to be deleted based on the parent status block set and the current allocated block set.

[0101] Further, the step of determining the block to be deleted based on the parent state block set and the currently allocated block set includes:

[0102] Perform a difference operation on the parent state block set and the current allocated block set, and determine the LBA that exists in the parent state block set but does not exist in the current allocated block set as a deleted block;

[0103] Record the LBA and deletion marker of the deleted block in the current backup image.

[0104] The deletion of blocks is determined based on the difference between the parent CRC Snapshot and the current block state baseline. The program uses the block-level LBA set aligned with the current Bitmap as the current allocated block set, and the LBAs recorded in the parent CRC Snapshot as the parent state block set. LBAs present in the parent state block set but not in the current allocated block set are identified as deleted blocks that have been released, and a deletion marker is recorded in the current backup image. When generating a new CRC Snapshot, the program uses the parent CRC Snapshot as a basis, overwrites the changed block checksums, and removes the entries corresponding to the deleted blocks.

[0105] Furthermore, it also includes a tiered degradation processing step:

[0106] When the master file table snapshot in the parent status information does not exist, the current master file table information reading fails, or the candidate file's data execution list cannot be completely parsed or the file record cannot be conservatively mapped, the step of comparing the master file table snapshot with the current master file table information is skipped. All allocated blocks in the current block-level LBA baseline are taken as candidate change ranges, and check values ​​are calculated block by block and compared with the cyclic redundancy check snapshot in the parent status information to confirm the real changed blocks.

[0107] When reading the volume allocation bitmap from the master file table path fails, the system volume allocation bitmap query interface is called to obtain the volume allocation status.

[0108] When the system volume allocation bitmap query interface fails to call but the total volume size can be obtained, a backup block range is formed based on the entire volume range.

[0109] This method employs a tiered degradation process. In normal mode, the program uses a Bitmap block state baseline, MFT candidate location, and CRC snapshot verification to confirm change detection. If the parent MFT snapshot does not exist, the current MFT read fails, Data Runs cannot convert candidate LBAs, or a file record that cannot be conservatively mapped is detected, the program enters level one degradation, skipping MFT candidate location and performing a full CRC scan on all allocated blocks in the current block state baseline. If reading $Bitmap via MFT fails, the program enters level two degradation, using the system Bitmap query interface to obtain the allocation status. If the system interface still fails but the volume size can be obtained, the program enters level three degradation, forming a backup block range based on the entire disk. The above degradation only expands the scan range or reduces backup efficiency; it does not change the process of confirming changes and generating subsequent parent states through CRC snapshots.

[0110] The VSS and NTFS hot backup block change detection method of this embodiment is applicable to online hot backup and subsequent incremental change detection of NTFS formatted disk volumes under the Windows operating system environment. This method is executed by a backup program running on a backup server or backup terminal, which interacts with the source volume through a programming interface provided by the operating system. In this embodiment, a 2TB NTFS formatted data volume with a cluster size of 4KB is used as the volume to be backed up, and the preset backup block size is 64KB (i.e., one backup block covers 16 clusters). The complete implementation process of this method is described below.

[0111] It should be noted that the backup block size can be dynamically adjusted based on the total capacity of the volume to be backed up and the expected backup granularity. As a preferred implementation, the backup block size is determined according to the following rules: when the total capacity of the volume to be backed up is less than or equal to 500GB, the backup block size is 32KB; when the total capacity of the volume to be backed up is greater than 500GB but less than or equal to 2TB, the backup block size is 64KB; and when the total capacity of the volume to be backed up is greater than 2TB, the backup block size is 128KB. Using this dynamic determination strategy, a reasonable total number of backup blocks can be maintained on volumes of different capacities, avoiding excessive CRCSnapshot metadata usage due to an excessive number of backup blocks, and also avoiding overly coarse granularity in single-change detection due to excessively large backup blocks.

[0112] like Figure 2 As shown, the specific process of this embodiment includes the following steps:

[0113] Step A1: Read the file system metadata and build a block state baseline.

[0114] like Figure 3 As shown, the backup program first opens the NTFS volume to be backed up, reads the volume's boot sector (DBR), and parses the following file system metadata parameters from the DBR: bytes per sector, sectors per cluster, bytes per cluster, MFT starting cluster number, and MFT record size. The bytes per cluster is obtained by multiplying the bytes per sector by the sectors per cluster, and is used for subsequent conversions between cluster numbers and byte offsets. These parameters form the basic coordinate system for all subsequent operations.

[0115] After obtaining the above parameters, the backup program reads the volume allocation bitmap to obtain the set of allocated clusters in the volume. In this embodiment, the volume allocation bitmap is read using a strategy of MFT path priority and system interface alternatives:

[0116] First, the backup program reads the volume allocation bitmap via the MFT path. Specifically, in the NTFS file system, the volume allocation bitmap corresponds to the metafile $Bitmap, and its record number in the MFT is 6. The backup program calculates the starting byte offset of the MFT in the volume based on the starting cluster number of the MFT obtained from the DBR, and reads the file record with record number 6 in the MFT. It parses the $DATA attribute (i.e., the data attribute, with an attribute type identifier of 0x80) in this file record to determine whether the attribute is resident or non-resident. If it is a resident attribute, the attribute header directly contains the data content of the volume allocation bitmap; if it is a non-resident attribute, it parses the data run list associated with the attribute. The data run list describes the physical storage location of the volume allocation bitmap data content in the volume in the format of "starting cluster number, cluster number". The backup program reads the complete content of the volume allocation bitmap from the volume according to the cluster range indicated by the data run list.

[0117] Logically, the volume allocation bitmap is an array of bits, where each bit corresponds to a cluster in the volume. A bit value of 1 indicates that the corresponding cluster has been allocated to a file or file system metadata, while a bit value of 0 indicates that the corresponding cluster is currently unallocated. The backup program iterates through each bit in the volume allocation bitmap, recording the cluster numbers of all clusters with a bit value of 1 as the set of allocated clusters.

[0118] If the MFT path read fails, the backup program enters the first-level degradation path and uses the system-provided volume allocation bitmap query interface to obtain the allocation status. Specifically, the backup program calls the Windows API function DeviceIoControl, passing in the FSCTL_GET_VOLUME_BITMAP control code. The file system driver returns a bit array indexed by the logical cluster number, which the backup program uses to parse out the set of allocated clusters.

[0119] If the system interface call still fails (e.g., the driver does not support the control code or returns an error), but the backup program can obtain the total number of sectors and clusters of the volume through interfaces such as GetDiskFreeSpace, then it enters the second-level degradation path. In this path, all clusters from cluster 0 to the largest cluster number in the volume are considered allocated clusters, meaning the entire volume range is used as the baseline for subsequent processing. Under this degradation path, although unallocated space cannot be skipped and the amount of backup data increases, the backup process will not be interrupted due to the inability to obtain the volume allocation bitmap, ensuring the completion of the backup task.

[0120] Step A2: Convert the allocated cluster set into an LBA baseline at the backup block granularity.

[0121] The allocated cluster set obtained in step A1 is cluster-level. Since the subsequent block indexing, cyclic redundancy check snapshot, and change detection processes in this method are all based on backup blocks, it is necessary to aggregate the cluster-level allocation status into an LBA baseline at the backup block level.

[0122] The backup program divides the volume's logical space into contiguous backup blocks according to a preset backup block size (64KB in this embodiment). Each backup block covers multiple contiguous clusters (16 clusters in this embodiment). For a given backup block, the backup program checks whether at least one cluster within the cluster range covered by the backup block belongs to the allocated cluster set determined in step A1; if so, the LBA of the backup block is included in the block-level LBA baseline.

[0123] Through the above aggregation process, the backup program converts the cluster-level volume allocation bitmap information into a backup block-level LBA coordinate system. The resulting block-level LBA baseline serves as the basis for determining which data blocks to read and write during subsequent full backups, and also as the reference coordinate for judging block status changes during subsequent incremental backups.

[0124] As a preferred implementation, during the aggregation process, the backup program also records the percentage of allocated clusters within the coverage area of ​​each backup block. This percentage information can be used for subsequent optimization—for example, in degraded mode, for backup blocks with a low percentage of allocated clusters (e.g., below 25%), sparse reads and writes can be performed at the cluster level rather than the block level to reduce the amount of invalid data transferred.

[0125] Step A3: Create a VSS snapshot and read the backup data based on the LBA benchmark.

[0126] The backup program calls the Windows VSS service to create a VSS snapshot of the NTFS volume to be backed up. The VSS snapshot creation process is as follows: the backup program sends a snapshot creation request to the VSS coordinator, the VSS coordinator notifies each writer on the volume to temporarily freeze write operations, creates the snapshot after the file system reaches a consistent state, and then resumes writing. The entire snapshot creation process is usually completed within a few seconds, with minimal impact on the business system. VSS snapshots operate based on a copy-on-write mechanism: after a snapshot is created, if the business system performs a write operation on a data block in the source volume, VSS copies the original content of the data block to the snapshot storage area before the write occurs, and then allows new data to be written to the source volume. Therefore, throughout the snapshot's lifecycle, regardless of how many writes occur to the source volume, the backup program always reads the data content at the time of snapshot creation from the VSS snapshot.

[0127] After a snapshot is successfully created, the backup program reads the data content corresponding to each backup block from the VSS snapshot sequentially, according to the block-level LBA baseline generated in step A2. For each backup block, the read data is written to the corresponding offset position in the backup image file. The backup image file is stored in the backup target storage, and its internal structure is arranged according to the LBA order of the backup blocks. Each backup block occupies a fixed amount of storage space in the image.

[0128] It should be noted that for backup blocks not included in the block-level LBA baseline, the backup program does not perform actual data reading, but only retains the corresponding position in the image (the data at this position can be filled with zeros or marked as unallocated), thus skipping invalid data blocks with unallocated space during the full backup stage, reducing the amount of backup data and reading overhead.

[0129] Step A4: Generate and save the parent state information.

[0130] After the full backup data is written, the backup program generates three types of parent status information based on the written backup data: block index, cyclic redundancy check snapshot, and master file table snapshot.

[0131] The block index records the storage location information of each backed-up data block in the backup image, including: the LBA of the backup block, the starting offset and data length of the backup block data in the backup image file, and the verification information of the backup block. The block index is used to quickly locate the data content corresponding to a specific LBA in the image during subsequent data recovery, and also to query whether the data of a certain LBA in the parent backup is available during incremental backup.

[0132] Cyclic Redundancy Check (CRC) snapshots are recorded at the backup block level, recording the CRC checksum value of the data block corresponding to each backed-up LBA. In this embodiment, the CRC checksum is calculated using the CRC32 algorithm (32-bit CRC algorithm), taking the entire data content of the backup block as input and outputting a 32-bit checksum. In this embodiment, the CRC32 algorithm uses the IEEE 802.3 standard polynomial 0x04C11DB7, with an initial value of 0xFFFFFFFF and an XOR value of 0xFFFFFFFF. Both the input and output are bit-reversed. As an optional optimization method, when the backup block size is large (e.g., 128KB), the CRC64 algorithm (a 64-bit cyclic redundancy check algorithm, using the ECMA-182 standard polynomial 0x42F0E1EBA9EA3693, with an initial value of 0x00000000000000000, and an XOR value of 0x000000000000000000) can be used to improve the distinguishability of the checksum and reduce the probability of different data blocks generating the same CRC value. Cyclic redundancy check snapshots are stored in a segmented contiguous storage format: the checksums of all LBAs in the same backup chain are stored consecutively in ascending LBA order, and the starting LBA, the number of LBAs, and the number of bytes occupied by each checksum are recorded at the beginning of the file. This storage format supports directly calculating the offset of the checksum in the file using LBAs, achieving constant-time random access and avoiding the overhead of loading the complete cyclic redundancy check snapshot into memory during incremental backups.

[0133] The Master File Table (MFT) snapshot records the file metadata information of the NTFS volume's Master File Table at the time of the full backup. Specifically, the backup program reads the MFT from the VSS snapshot, traverses each file record in the MFT (each file record is typically 1024 bytes), and extracts the following information: file reference number, parent directory file reference number, file flags (including file record usage status flags), file creation time, file modification time, file MFT modification time, file access time, file data size, file attribute flags (including read-only, hidden, system, compressed, encrypted, sparse, etc.), and a data run list summary of the file.

[0134] It is important to note that the Master File Table snapshot records a summary of the data run list, not the complete data run list. The data run list summary is generated by concatenating each "starting cluster number, cluster number" entry in the data run list into a byte sequence according to its order in the list, calculating a hash value (such as a SHA-1 hash) on this byte sequence, and storing the hash value as the summary. Using a summary instead of storing the entire list significantly reduces the storage space of the Master File Table snapshot (for large volumes containing millions of files, the complete storage of the data run list can occupy hundreds of MB or even GB of space). During subsequent MFT comparisons, the same hash algorithm is calculated on the current data run list, and the summary value is compared to efficiently determine whether the data run list has changed. Those skilled in the art will understand that this method trades a very small probability of summary collisions (the collision probability of SHA-1 is negligible in practical engineering) for significant storage space savings.

[0135] The block index, cyclic redundancy check snapshot, and master file table snapshot are collectively stored as a parent state for use in subsequent incremental backups. Specifically, the master file table snapshot is used to locate candidate changes during subsequent backups, the cyclic redundancy check snapshot is used to confirm block-level changes during subsequent backups, and the block index is used for locating mirrored data and data recovery.

[0136] Step A5: Locate candidate changes based on MFT Snapshot.

[0137] like Figure 4 As shown, when incremental backup is required, the backup program first reads the cyclic redundancy check snapshot and master file table snapshot from the parent status information, and creates a VSS snapshot of the current volume to be backed up, and reads the current MFT information from the current VSS snapshot.

[0138] The backup program compares the master file table snapshot with the current MFT information line by line. The comparison is indexed by file reference number, and the following comparison operations are performed on file records with the same file reference number in the parent snapshot and the current MFT:

[0139] First, compare the usage status flags of the file records. If the record is in use in the parent snapshot but not in the current MFT, the file is considered a candidate for deletion; if the file reference number does not exist in the parent snapshot but exists in the current MFT, the file is considered a candidate for addition.

[0140] Second, compare file modification times. Each file record in the NTFS file system contains multiple timestamps, among which the file modification time (corresponding to the modification time field in the $STANDARD_INFORMATION attribute) records the time when the file data was last modified. If the file modification time in the current MFT is later than the modification time recorded in the parent snapshot, the file is considered a candidate file that may have been modified. It should be noted that, as a preferred implementation, a tolerance threshold (e.g., 1 to 2 seconds) is introduced when comparing modification times to eliminate boundary misjudgments caused by differences in file system timestamp precision or backup time points. The tolerance threshold of 1 to 2 seconds is based on the fact that while the nominal precision of NTFS file timestamps is 100 nanoseconds, there is a certain system call delay and clock skew between the update of file timestamps and the reading by the backup program in the actual operating system. A tolerance window of 1 to 2 seconds can cover most boundary situations caused by system scheduling delays and clock synchronization skews, while avoiding the possibility of missing file modifications that actually occurred within one or two seconds due to excessive tolerance.

[0141] Third, compare file data sizes. If the file data size recorded in the current MFT differs from that recorded in the parent snapshot, the file is considered a candidate file that may have been modified. Changes in data size usually indicate that file content has been appended or truncated.

[0142] Fourth, compare the data run list digests. As mentioned earlier, calculate the same hash algorithm as in step A4 for the data run list of the current file, and compare the resulting hash value with the data run list digest stored in the parent snapshot. If the digest values ​​are different, it indicates that the physical storage layout of the file's data on the disk has changed (e.g., the file has been reassigned to a storage cluster), and the file is identified as a candidate changed file.

[0143] The results of the above four comparisons are logically ORed: if any one of them is satisfied, the file is included in the candidate change file set. The backup program parses the data run list of each file in the candidate change file set, extracts the cluster range from each "starting cluster number, cluster number" entry, converts the cluster range into LBA, and then aligns them according to the backup block size to obtain the candidate change LBA for that file. The LBAs generated from multiple files or multiple data run list entries are merged and deduplicated to obtain the final candidate change LBA set.

[0144] In the merging and deduplication of candidate LBAs, the backup program employs a range merging algorithm: first, all candidate LBAs are sorted in ascending order; then, the sorted sequence is traversed, merging adjacent or overlapping LBA ranges. For example, if the candidate set contains LBAs 100 to 105 and LBAs 104 to 110, they are merged into LBAs 100 to 110. Using range merging reduces the need for repeated CRC calculations and reads on the same backup block range in subsequent iterations.

[0145] It's also important to note that for sparse files in the NTFS file system, their data run lists may contain empty regions where no physical clusters have actually been allocated (represented by a special zero-length cluster marker in the data run list). When parsing the data run list, the backup program identifies and skips the cluster ranges corresponding to these empty regions, excluding them from the candidate change LBA set. The detection criteria for empty regions in sparse files are: in the data run list, if the difference between the starting cluster numbers of two adjacent entries is greater than the cluster number of the destination of the previous entry, it indicates that there is an empty interval between them where no physical storage has been allocated.

[0146] Step A6: Confirm the candidate change LBA based on the CRC Snapshot.

[0147] The candidate change LBA set generated in step A5 is determined based on MFT metadata changes. This set may contain a large number of LBAs where the actual data content has not changed. Examples include situations where only file attributes (such as read-only flags or access times) have been modified while the data content remains unchanged, or where files have been moved within the same volume, resulting in a change in the data run list summary but not the data block content. Therefore, each candidate change LBA needs to be verified at the content level.

[0148] The backup program iterates through each LBA in the candidate changed LBA set and performs the following processing: It reads the data content of the backup block corresponding to the LBA from the current VSS snapshot and calculates its CRC checksum (using the same CRC algorithm and parameters as when generating the Cyclic Redundancy Check snapshot in step A4); it reads the historical CRC checksum corresponding to the LBA from the parent Cyclic Redundancy Check snapshot. The currently calculated CRC checksum is compared with the historical CRC checksum. If they are different, the data block corresponding to the LBA is determined to be a real changed block, and this data block needs to be written to the current backup image. If they are the same, it means that although the MFT metadata has changed, the actual content of the data block has not changed, and the data block corresponding to the LBA does not need to be written to the current backup image.

[0149] Through the aforementioned CRC verification mechanism, MFT candidate localization narrows the change detection scope from the entire volume to a candidate LBA set, while CRC verification further filters out the blocks from the candidate LBA set that actually show changes in data content. The two work collaboratively: MFT provides the candidate range, and CRC provides content verification.

[0150] Step A7: Identify the deleted block and generate a new CRC Snapshot.

[0151] After identifying the actual changed blocks, the backup program also needs to identify the data blocks in this backup round that need to be marked as deleted. The logic for identifying deleted blocks is based on a comparison between the parent cyclic redundancy check snapshot and the current volume allocation bitmap:

[0152] First, the backup program obtains the current volume allocation bitmap and aggregates it into a block-level LBA set in the same manner as in step A2, which serves as the current allocated block set. Then, it extracts all LBAs recorded in the parent cyclic redundancy check snapshot, forming the parent state block set. The parent state block set represents all data blocks that existed in the volume at the time of the previous backup. A difference operation is performed between the parent state block set and the current allocated block set: LBAs that exist in the parent state block set but not in the current allocated block set are the deleted blocks that have been released in this backup. The data corresponding to these deleted blocks existed in the parent backup, but the corresponding clusters at the current time have been reclaimed by the file system and are in an unallocated state.

[0153] The backup program records a deletion marker for each deleted block in the current backup image. This deletion marker includes the LBA of the deleted block and the deletion timestamp, so that the data state at each point in time can be accurately restored during data recovery.

[0154] After writing the actual changed blocks and marking the deleted blocks, the backup program generates a new cyclic redundancy check (CRRC) snapshot for this round of backups. The generation of the new CRRC snapshot is based on the parent CRRC snapshot, performing the following three types of update operations: For LBAs that exist in the parent CRRC snapshot but do not belong to the deleted blocks or the actual changed blocks, their CRC checksums remain unchanged; for LBAs corresponding to actual changed blocks, the current CRC checksum calculated in step A6 is used to overwrite the original checksum; for LBAs corresponding to deleted blocks, their entries are removed from the CRRC snapshot. Through these operations, the newly generated CRRC snapshot fully reflects the block-level checksum status after this round of backups is completed and can serve as the parent status for the next round of incremental backups.

[0155] A key feature of this method is that after each round of backups is completed, a complete, self-contained cyclic redundancy check (CRUD) snapshot is generated, rather than simply recording incremental changes. This design allows any subsequent incremental backup to obtain the complete parent state simply by reading the most recent CRUD snapshot, eliminating the need to backtrack and merge the check information from multiple incremental backups, thereby reducing the management complexity of the incremental backup chain.

[0156] Step A8: Layered degradation processing.

[0157] This method establishes a complete tiered degradation mechanism to ensure that the backup task can continue to complete even when various anomalies are encountered during the backup process, rather than failing and exiting directly. Degradation processing is divided into three tiers:

[0158] In normal mode, the backup process uses the complete processing path: building a block state baseline with the volume allocation bitmap, narrowing down the change detection range with MFT candidate positioning, and performing verification with CRC Snapshot. This is the most efficient normal path.

[0159] The backup program enters Level 1 degradation when any of the following abnormal conditions occur: the master file table snapshot does not exist, the current MFT read fails, the data run list of candidate files cannot be completely parsed, or a file record in the MFT that cannot be conservatively mapped is detected. Under Level 1 degradation, the backup program skips the MFT candidate location step, treats all allocated blocks in the current block state baseline as candidate change ranges, performs CRC checksum calculations on each of these blocks, and compares them with the parent cyclic redundancy check snapshot. Although Level 1 degradation expands the range requiring block-by-block verification, the change detection logic confirmed by CRC snapshot verification remains unchanged, and both truly changed blocks and deleted blocks can be correctly identified.

[0160] The second and third levels of degradation have been detailed in step A1: when the MFT path fails to read the volume allocation bitmap, it enters the second level of degradation and uses the system volume allocation bitmap query interface instead; when the system interface also fails but the volume size can be obtained, it enters the third level of degradation and uses the entire volume range as the backup block range.

[0161] The three-level degradation process described above constitutes the layered protection system of this method. From the first level to the third level, the degradation degree gradually deepens, the backup scan scope gradually expands, and the backup efficiency gradually decreases. However, each level of degradation maintains the core capability of confirming changes and generating subsequent parent states through CRC snapshots. This means that even in the most severe degradation scenario (third-level degradation—full volume scan), the current round of backup can still be completed, and the newly generated cyclic redundancy check snapshot can still serve as the parent state for the next round of incremental backups. The backup chain will not be broken due to a single anomaly.

[0162] As a further optimization measure, the backup program records the time of each downgrade, the downgrade level, the specific reason for triggering the downgrade, and the impact indicators of the downgrade on backup performance in the log, so that operations and maintenance personnel can subsequently analyze and deal with the root cause of the downgrade.

[0163] Secondly, this application proposes a VSS and NTFS hot backup block change detection system, such as... Figure 5 As shown, it includes:

[0164] The block status baseline construction module is used to read the volume allocation bitmap of the NTFS volume to be backed up, determine the allocated cluster set, and aggregate the allocated cluster set into a block-level LBA baseline according to the preset backup block size.

[0165] The VSS snapshot read module is used to create a VSS snapshot of the NTFS volume to be backed up as a consistent read point, read data blocks from the VSS snapshot according to the block-level LBA baseline, and write the read data blocks to the backup image.

[0166] The parent state generation module is used to generate block indexes, cyclic redundancy check snapshots, and master file table snapshots as parent state information;

[0167] The candidate change location module is used to compare the master file table snapshot with the master file table information in the current VSS snapshot during subsequent backups, filter candidate change files, and convert the data run list of candidate change files into a candidate change LBA set.

[0168] The change confirmation module is used to read the corresponding data block from the current VSS snapshot and calculate the check value for LBA in the candidate change LBA set. The calculated check value is compared with the corresponding LBA check value in the cyclic redundancy check snapshot in the parent status information. LBA with different check values ​​are determined to be real change blocks.

[0169] The deletion block identification module is used to take the block-level LBA set after the current volume allocation bitmap is aligned as the current allocated block set, take the LBA recorded in the cyclic redundancy check snapshot in the parent status information as the parent status block set, and determine the deletion block based on the parent status block set and the current allocated block set.

[0170] The tiered degradation processing module is used to perform tiered degradation processing when anomalies occur in the master file table snapshot path, volume allocation bitmap read path, or data run list parsing path, in order to maintain the continuity of the backup process by expanding the scan scope.

[0171] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0172] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0173] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0174] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0175] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0176] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0177] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0179] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0180] The above are merely preferred embodiments of the present invention. It should be noted that any modifications and improvements made by those skilled in the art without departing from the present technical solution should also be considered to fall within the scope of protection claimed by the present solution.

Claims

1. A method for detecting changes in VSS and NTFS hot backup blocks, characterized in that, Includes the following steps: Read the volume allocation bitmap of the NTFS volume to be backed up, determine the allocated cluster set, and aggregate the allocated cluster set into a block-level LBA baseline according to the preset backup block size; Create a Volume Shadow Copy Service (VSS) snapshot of the NTFS volume to be backed up as a consistent read point. Read data blocks from the VSS snapshot according to the block-level LBA baseline, write the read data blocks to the backup image, and generate block index, cyclic redundancy check snapshot and master file table snapshot as parent status information. During subsequent backups, the master file table snapshot is compared with the master file table information in the current VSS snapshot to filter candidate changed files, and the data run list of the candidate changed files is converted into a candidate changed LBA set; For LBAs in the candidate change LBA set, read the corresponding data block from the current VSS snapshot and calculate the check value. Compare the calculated check value with the corresponding LBA check value of the cyclic redundancy check snapshot in the parent status information. LBAs with different check values ​​are determined to be real change blocks. The block-level LBA set aligned with the current volume allocation bitmap is used as the current allocated block set, and the LBA recorded in the cyclic redundancy check snapshot in the parent status information is used as the parent status block set. The block to be deleted is determined based on the parent status block set and the current allocated block set.

2. The method according to claim 1, characterized in that, The process of reading the volume allocation bitmap of the NTFS volume to be backed up includes: The volume allocation bitmap is read first through the master file table path: the meta file record corresponding to the volume allocation bitmap is located according to the starting position of the master file table, the data attributes and data run list in the meta file record are parsed, and the volume allocation bitmap content is read according to the cluster range indicated by the data run list to obtain the allocated cluster set; If reading the main file table path fails, the system volume allocation bitmap query interface is called to obtain the volume allocation status. If the system volume allocation bitmap query interface call fails, but the total volume size can be obtained, then the backup block range is formed based on the entire volume range.

3. The method according to claim 2, characterized in that, The step of preferentially reading the volume allocation bitmap via the master file table path includes: Read the boot sector of the NTFS volume to be backed up and obtain the starting cluster number of the master file table; The file record in the master file table is located based on the starting cluster number of the master file table, and the record number of the file record is 6; The data attributes of the file record are parsed. If the data attribute is a resident attribute, the volume allocation bitmap content is read directly from the attribute header. If the data attribute is a non-resident attribute, the data run list is parsed, and the volume allocation bitmap content is read from the volume according to the starting cluster number and the number of clusters in the data run list.

4. The method according to claim 3, characterized in that, The process of aggregating the allocated cluster set into a block-level LBA baseline according to a preset backup block size includes: The logical space of the volume to be backed up is divided into contiguous backup blocks according to the preset backup block size, and each backup block covers multiple clusters; For each backup block, check whether there is at least one cluster within its covered cluster range that belongs to the allocated cluster set; If it exists, the LBA of the backup block will be included in the block-level LBA baseline.

5. The method according to claim 4, characterized in that, The process of comparing the master file table snapshot with the master file table information in the current VSS snapshot to filter candidate changed files includes: Using the file reference number as an index, for file records with the same file reference number in the main file table snapshot and the current main file table information, compare the file record's usage status flag, file modification time, file data size, and data execution list summary; If the usage status flag of a file record changes from "in use" to "unused", it is determined to be a candidate file for deletion; if a file reference number exists in the current main file table information but does not exist in the main file table snapshot, it is determined to be a candidate file for addition. If a file is modified later than the modification time recorded in the master file table snapshot, the file data size is different from that recorded in the master file table snapshot, or the data run list summary is different from that recorded in the master file table snapshot, then it is determined to be a candidate file that may have been modified.

6. The method according to claim 5, characterized in that, The checksums of the LBAs in the candidate change LBA set are compared, including: Read the data content of the backup block corresponding to the candidate change LBA from the current VSS snapshot, and calculate the check value using the same cyclic redundancy check algorithm as when generating the cyclic redundancy check snapshot; Read the historical parity value of the corresponding LBA from the Cyclic Redundancy Check snapshot in the parent status information; If the currently calculated checksum is different from the historical checksum, the data block corresponding to the LBA is determined to be a real changed block and written to the current backup image; if they are the same, the data block is determined to have no content change and is not written to the current backup image.

7. The method according to claim 6, characterized in that, The step of determining the block to be deleted based on the parent state block set and the currently allocated block set includes: Perform a difference operation on the parent state block set and the current allocated block set, and determine the LBA that exists in the parent state block set but does not exist in the current allocated block set as a deleted block; Record the LBA and deletion marker of the deleted block in the current backup image.

8. The method according to claim 7, characterized in that, The creation of the Volume Shadow Copy Service (VSS) snapshot of the NTFS volume to be backed up serves as a consistent read point. Data blocks are read from the VSS snapshot based on block-level LBA benchmarks, including: Initiate a snapshot creation request to the volume shadow copy service to generate a consistent data view of the NTFS volume to be backed up at the time of snapshot creation; When the business system continuously writes to the source volume, the data content at the time of snapshot creation is read through the write-on-write replication mechanism of VSS snapshot, so that all data blocks read during the backup process belong to the same point in time.

9. The method according to claim 8, characterized in that, It also includes a tiered degradation process: When the master file table snapshot in the parent status information does not exist, the current master file table information reading fails, the data running list of the candidate file cannot be completely parsed, or a file record that cannot be conservatively mapped is detected, the step of comparing the master file table snapshot with the current master file table information is skipped. All allocated blocks in the current block-level LBA baseline are taken as candidate change ranges, and check values ​​are calculated block by block and compared with the cyclic redundancy check snapshot in the parent status information to confirm the real changed blocks. When reading the volume allocation bitmap from the master file table path fails, the system volume allocation bitmap query interface is called to obtain the volume allocation status. When the system volume allocation bitmap query interface fails to call but the total volume size can be obtained, a backup block range is formed based on the entire volume range.

10. A VSS and NTFS hot backup block change detection system, characterized in that, include: The block status baseline construction module is used to read the volume allocation bitmap of the NTFS volume to be backed up, determine the allocated cluster set, and aggregate the allocated cluster set into a block-level LBA baseline according to the preset backup block size. The VSS snapshot read module is used to create a VSS snapshot of the NTFS volume to be backed up as a consistent read point, read data blocks from the VSS snapshot according to the block-level LBA baseline, and write the read data blocks to the backup image. The parent state generation module is used to generate block indexes, cyclic redundancy check snapshots, and master file table snapshots as parent state information; The candidate change location module is used to compare the master file table snapshot with the master file table information in the current VSS snapshot during subsequent backups, filter candidate change files, and convert the data run list of candidate change files into a candidate change LBA set. The change confirmation module is used to read the corresponding data block from the current VSS snapshot and calculate the check value for LBA in the candidate change LBA set. The calculated check value is compared with the corresponding LBA check value in the cyclic redundancy check snapshot in the parent status information. LBA with different check values ​​are determined to be real change blocks. The deletion block identification module is used to take the block-level LBA set after the current volume allocation bitmap is aligned as the current allocated block set, take the LBA recorded in the cyclic redundancy check snapshot in the parent status information as the parent status block set, and determine the deletion block based on the parent status block set and the current allocated block set. The tiered degradation processing module is used to perform tiered degradation processing when anomalies occur in the master file table snapshot path, volume allocation bitmap read path, or data run list parsing path, in order to maintain the continuity of the backup process by expanding the scan scope.

Citation Information

Patent Citations

  • Incremental backing up method and system based on data block

    CN104375905A

  • Data backup system and method for NTFS file system

    CN111414280A

  • Method for increasing NTFS incremental backup speed

    CN119883741A

  • Server-free backup method and system based on incremental bitmap between storage snapshots

    CN106126370A

  • Block-level data deduplication method based on NTFS file system

    CN112416879A