A distributed file system-based incremental backup method, backup device, medium and program product

Abstract

The application discloses a kind of based on distributed file system incremental backup method, backup device, medium and program product, and relate to distributed storage technical field.Through using the above technical scheme, since using the data change log based on the statistics each data block update frequency, and the data block is divided into low-frequency update set and high-frequency update set and the staged backup strategy of starting high-frequency block reading after low-frequency block is all read, the inconsistency risk caused by the time dispersion distribution of data block reading in backup file is reduced, and the internal consistency level of incremental backup data as a whole is improved.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and in particular to an incremental backup method, backup device, media, and program product based on a distributed file system. Background Technology

[0002] In modern enterprise-level data centers and cloud computing architectures, distributed file systems have become the core infrastructure for carrying massive amounts of business data due to their high scalability and high availability. As business scale continues to expand, data volume explodes, requiring regular backups of the file system to prevent data loss or corruption.

[0003] In related technologies, to address the resource consumption and time latency issues associated with full backups, incremental backup technology based on timeline order is typically employed. When a pre-defined backup task is triggered, the distributed file system retrieves all data change logs since the last backup was completed, parses them sequentially according to the chronological order of the log entries, and initiates data read requests to the source storage nodes in that order. The retrieved changed data blocks are then appended sequentially to the incremental backup file, thus achieving backup only of newly added or modified data.

[0004] However, to meet the demands of high-performance computing, distributed file systems often host virtual machine services that are bound to pass-through hardware (such as vGPUs). During operation, these services typically do not support traditional low-level storage snapshot functionality because they cannot perform memory state quiescence or I / O suspension. Furthermore, the backup program's cross-network fetching of incremental data usually incurs significant transmission time, and the access and modification frequencies of different data blocks within the file system vary significantly, with some core business data blocks being updated extremely frequently within a short period. When related technologies fetch data sequentially according to log time, the time difference between fetching time and continuous business writes may increase the risk of backup content degradation, thereby reducing the reliability of backup data and the operational stability of the distributed file system. Summary of the Invention

[0005] This application provides an incremental backup method, backup device, media, and program product based on a distributed file system to improve the consistency of backup data in scenarios without snapshots.

[0006] In a first aspect, this application provides an incremental backup method based on a distributed file system, applied to a backup device. The method includes: responding to a preset backup trigger condition, determining a data change log set within a target time window, the data change log set containing multiple data change log records, each containing logical address information of a data block that has undergone data changes; traversing the data change log set to determine the total number of times the logical address appears as a change object in the data change log set, and determining the total number of times as the update frequency value of the data block corresponding to the logical address; dividing the data block into a low-frequency update set and a high-frequency update set according to a preset frequency threshold; determining first incremental data corresponding to the low-frequency update set based on the logical address information of each data block in the low-frequency update set; after all the first incremental data has been acquired, determining second incremental data corresponding to the high-frequency update set based on the logical address information of each data block in the high-frequency update set; and writing the first incremental data and the second incremental data into an incremental backup file.

[0007] By adopting the above technical solution, the total number of occurrences of the logical address of each data block is counted by traversing the data change log set as the update frequency value. Based on this, data blocks are divided into low-frequency update sets and high-frequency update sets. The reading of high-frequency blocks is only started after all low-frequency blocks have been read. This compresses the time interval between the reading of high-frequency data blocks and their writing to the backup file to the final stage of the backup task. Since the probability of high-frequency data blocks being modified again by the business system within a unit of time is much higher than that of low-frequency blocks, postponing their reading to the end of the task shortens the time window during which the content of high-frequency blocks is exposed to business write operations during network transmission. This effectively narrows the time deviation between the data state recorded in high-frequency blocks and low-frequency blocks in the backup file. Thus, in scenarios where snapshot technology cannot be used to freeze the storage state, the risk of inconsistency caused by the discrete distribution of data block reading times within the backup file is reduced, improving the overall internal consistency level of incremental backup data.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the step of dividing the data block into a low-frequency update set and a high-frequency update set according to a preset frequency threshold specifically includes: obtaining the number of hard links associated with other directory entries for each inode to which the data block belongs, based on the file system metadata corresponding to the data block; if the arithmetic mean of the number of hard links is greater than a preset dependency threshold, calculating the probability that each data block and other data blocks will be concurrently written to disk in the same transaction log within a preset historical period, as the concurrency aggregation degree; and performing numerical regression analysis between the number of hard links and the concurrency aggregation degree. After unification, a weighted calculation is performed using preset weight parameters to obtain concurrent association weight coefficients that characterize the risk of each data block being associated with modification. By sorting the sequence of concurrent association weight coefficients corresponding to all data blocks according to their numerical values, the boundary value with the largest difference between adjacent weight coefficients is identified, and this boundary value is used as the updated preset frequency threshold. According to the preset frequency threshold, data blocks with update frequency values ​​less than the preset frequency threshold are assigned to the low-frequency update set, and data blocks with update frequency values ​​greater than or equal to the preset frequency threshold are assigned to the high-frequency update set.

[0009] By employing the aforementioned technical solution, the number of hard links to the index nodes of each data block is obtained and their arithmetic mean is calculated. When the mean exceeds a dependency threshold, it is determined that the current data block set has a multi-path reference structure. Furthermore, the probability of each data block being concurrently written to disk with other blocks in the historical transaction log is calculated as the concurrency aggregation degree. The number of hard links and the concurrency aggregation degree are normalized and weighted to form a concurrency association weight coefficient. The frequency threshold is then dynamically updated by extracting the boundary point with the largest adjacent difference in the ascending sequence of weight coefficients, ensuring that the threshold boundary naturally falls at the position where the differentiation between high and low association risks is most significant. Compared to a fixed threshold, this mechanism can accurately classify data blocks with a strong tendency for modification into the high-frequency set, preventing them from being read early in the backup process and causing logical state discontinuity within the backup data due to subsequent modifications of related blocks. This improves the accuracy of data block classification in perceiving association consistency risks.

[0010] In some embodiments of the first aspect, after obtaining the number of hard links associated with other directory entries for each data block's inode based on the file system metadata corresponding to the data block, the method further includes: if the arithmetic mean of the number of hard links is less than or equal to the preset dependency threshold, retrieving the physical block mapping table of the logical address corresponding to each data block on the physical storage medium; calculating the spatial offset of adjacent data blocks of each logical address on the physical storage medium, and defining the distribution ratio of data blocks with spatial offsets greater than a preset continuity threshold as spatial discrete sparsity; performing a weighted operation on the update frequency value and the spatial discrete sparsity to obtain the addressing index corresponding to each data block; establishing the boundary value when the difference between adjacent indices in the sequence formed by the addressing indices reaches the maximum value as an auxiliary threshold, and combining data blocks with addressing indices less than the auxiliary threshold into the low-frequency update set, and combining data blocks with addressing indices greater than or equal to the auxiliary threshold into the high-frequency update set.

[0011] By adopting the above technical solution, when the average number of hard links does not exceed the dependency threshold and the logical correlation of data blocks is low, the physical block mapping table is invoked instead to calculate the spatial offset of logically adjacent data blocks on the physical medium. The proportion of blocks with offsets exceeding the continuity threshold is defined as spatial dispersion sparsity—this indicator reflects the degree of fragmentation of data block distribution on the physical medium. Data blocks with high dispersion will trigger more random I / O during reading, resulting in longer reading times and an increased probability of secondary modification by business applications during this period. The spatial dispersion sparsity is weighted and fused with the update frequency value to form an addressing index, and the boundary value of the largest difference between adjacent sequences is used as an auxiliary threshold for classification. This incorporates the dual risks of physical layout degradation and update activity into the classification decision, thereby identifying data blocks with a high risk of content changes during backup reading and delaying their processing even in scenarios with insignificant logical correlations. This improves the adaptability of the classification mechanism to different storage characteristic scenarios.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, before the step of determining the second incremental data corresponding to the high-frequency update set based on the logical address information of each data block in the high-frequency update set after all the first incremental data has been acquired, the method further includes: within the time period of acquiring the first incremental data corresponding to the low-frequency update set, extracting the current network response latency and disk I / O wait queue length of each preferred high-frequency target server based on the preferred high-frequency target server corresponding to each data block in the high-frequency update set; calculating the node response latency multiplier of each preferred high-frequency target server based on the current network response latency and the disk I / O wait queue length; and performing candidate node addressing based on the target logical address corresponding to the preferred high-frequency target server that generates the node response latency multiplier, to determine the candidate server to replace the preferred high-frequency target server.

[0013] By employing the above technical solution, the current network response latency and disk I / O wait queue length of each preferred high-frequency target server are extracted in parallel during the low-frequency data block reading period, and the node response latency multiplier is calculated accordingly. Since low-frequency block reading itself requires an incompressible transmission time, node health assessment is nested within this waiting time, ensuring that the node pre-screening process does not additionally consume the critical path time of the backup task. When performance degradation of the preferred node is detected, alternative node addressing is immediately initiated, ensuring that the node switching decision is completed before the formal commencement of high-frequency block reading. This avoids the additional delays caused by discovering node anomalies and being forced to temporarily switch during the critical stage of high-frequency block reading, thus ensuring the controllable execution time of the high-frequency block reading phase and strengthening the main solution's ability to guarantee the consistency of high-frequency data blocks.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, the step of determining a replacement server for the preferred high-frequency target server by addressing alternative nodes based on the target logical address corresponding to the preferred high-frequency target server that generates the node response latency multiplier specifically includes: extracting all secondary candidate slave servers storing replicas of the data blocks corresponding to the target logical address through a metadata dictionary and the target logical address corresponding to the preferred high-frequency target server that generates the node response latency multiplier, and obtaining the latest replica synchronization timestamp of each secondary candidate slave server; after obtaining the last health timestamp of the preferred high-frequency target server, combining the secondary candidate slave servers whose replica synchronization timestamps are later than or equal to the last health timestamp into a secure alternative cluster; selecting the secondary candidate slave server with the highest historical average transmission rate from the secure alternative cluster as the alternative server; and updating the read request node corresponding to the target logical address to the alternative server.

[0015] By employing the above technical solution, all secondary candidate slave servers storing replicas of the target data block are located through a metadata dictionary. The replication timestamp of each server is obtained and compared with the last health timestamp of the preferred high-frequency target server. Only nodes with replication timestamps later than or equal to the last health timestamp are included in the safe backup cluster. The criterion of "whether the node replica contains all data prior to the failure" is used to exclude candidate nodes that might read outdated data due to replication synchronization delays. Within the safe backup cluster, selection is further optimized based on the historical average transmission rate, ensuring that high-frequency block readings after node switching are completed at the optimal rate without introducing the risk of data version rollback, thus guaranteeing the reading quality of the second incremental data.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of writing the first incremental data and the second incremental data into the incremental backup file, the method further includes: obtaining the delivery timestamp of the service response message sent by the alternative server to the backup device when transmitting the second incremental data; initiating an out-of-band alignment query request to the master control component of the distributed cluster corresponding to the target logical address based on the delivery timestamp; receiving feedback from the master control component that the master node has not issued a change log within the interval between the replica synchronization timestamp and the delivery timestamp; and, if the master node's unissued change log is not empty, determining that there is an unsynchronized data segment in the second incremental data, and triggering a data correction strategy for the unsynchronized data segment.

[0017] By adopting the above technical solution, after the second incremental data is written to the incremental backup file, the delivery timestamp of the business response message sent by the backup server is obtained. This timestamp, together with the replication synchronization timestamp of the backup server, forms a query interval. An out-of-band alignment query is then initiated to the main control component to request change logs that have been executed by the master node but not yet distributed to the backup server within this interval. This query occurs after data transmission is complete and does not interfere with the execution rhythm of the main backup process, constituting a lightweight post-verification mechanism. When the returned undistributed change logs are not empty, it is determined that there are unsynchronized data fragments in the backup file and correction is triggered, reducing the risk of recovery failure due to inconsistencies in replica versions during node switching scenarios.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, the step of triggering the data correction strategy for the unsynchronized data fragment specifically includes: extracting the physical offset range corresponding to the second incremental data in the incremental backup file as the data area to be repaired; extracting the missing data fragments that have not been synchronized to the alternative server within the range and the corresponding intra-block offset address of the missing data fragments according to the change logs not issued by the master node; locating the corresponding old data stream in the data area to be repaired according to the intra-block offset address; performing a data content overwrite operation on the old data stream using the missing data fragments, and calculating the block checksum based on the overwritten data area to be repaired, the block checksum being used to update the integrity verification information of the incremental backup file.

[0019] By employing the above technical solution, the physical offset range of the second incremental data in the incremental backup file is located using index information as the data area to be repaired. Then, missing data fragments and their offset addresses within blocks are extracted from the master node's unissued change logs. The old data stream is precisely located within the data area to be repaired based on the offset address, and byte-level overwriting is performed. Only the actual missing incremental changes are injected into the backup file, minimizing the data transfer volume of the correction operation. After overwriting, the block checksum is recalculated, and the integrity verification information of the backup file is updated, ensuring the verifiability of the correction result. The entire correction process completes a closed loop at the local file level, restoring the data version gap caused by replica synchronization deviation while maintaining the integrity and verification consistency of the incremental backup file structure.

[0020] In a second aspect, this application provides a backup device comprising: one or more processors and a memory; the memory is coupled to the one or more processors and is used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the backup device to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, this application provides a computer program product, including a computer program that, when executed by a processor, causes the network management device to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a backup device, cause the backup device to perform the method described in the first aspect and any possible implementation thereof.

[0023] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0024] 1. By adopting the above technical solution, the phased backup strategy, which uses data change logs to count the update frequency of each data block, divides the data blocks into low-frequency update sets and high-frequency update sets, and only starts reading high-frequency blocks after all low-frequency blocks have been read, effectively solves the technical problem in the existing technology that when pulling data one by one in log time order, the high-frequency data block reading takes a long time and is continuously exposed to business write operations during the transmission process, resulting in inconsistent data status inside the backup file.

[0025] 2. By adopting the above technical solution, the spatial dispersion sparsity is calculated based on the physical block mapping table when the logical correlation between data blocks is low. This sparsity is then weighted and fused with the update frequency value to form an addressing index. The classification threshold is adaptively determined by the maximum limit value of the adjacent sequence difference. This effectively solves the technical problem in the existing technology that the fixed frequency threshold cannot detect the degree of physical storage fragmentation of data blocks, which leads to high read time-consuming data blocks being incorrectly classified into low-frequency sets and read first in the early stage of backup, thereby amplifying the exposure time of their content change risk. Thus, it achieves the technical effect of incorporating the dual risks of physical layout degradation and update activity into the classification decision in scenarios where the logical correlation is not significant, identifying data blocks with high content change risk during backup reading and delaying their processing.

[0026] 3. By adopting the above technical solution, the pre-evaluation mechanism of monitoring the node response latency ratio of the preferred high-frequency target server in parallel during the low-frequency block reading time period and completing the alternative node addressing and read request node switching in advance when performance degradation is detected effectively solves the technical problem in the existing technology that node anomalies are only discovered in the critical stage of high-frequency block reading, forced temporary switching leads to increased reading latency, and thus expands the time window of high-frequency blocks exposed to business write operations. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating an incremental backup method based on a distributed file system in an embodiment of this application.

[0028] Figure 2 This is another flowchart illustrating an incremental backup method based on a distributed file system in an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of an exemplary hardware structure of the backup device in an embodiment of this application. Detailed Implementation

[0030] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0031] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0032] To facilitate understanding, the relevant terms and concepts involved in the embodiments of this application will be introduced below.

[0033] (1) Virtual Graphics Processing Unit (vGPU) refers to a virtualization implementation that allocates physical GPU resources to virtual machines through hardware passthrough technology, enabling virtual machines to directly access physical GPU hardware to obtain high-performance graphics rendering and parallel computing capabilities. In a virtualization environment, vGPU directly maps the computing units and video memory resources of the physical GPU to the virtual machine address space through PCIe device passthrough. Applications within the virtual machine can directly call the GPU driver to execute graphics rendering or parallel computing tasks. Because this passthrough mode bypasses the software abstraction of the virtualization layer, the memory state of the virtual machine is tightly coupled with the state of the GPU device. When the virtualization platform attempts to perform a snapshot operation, it needs to freeze the virtual machine memory, CPU registers, and the running state of the GPU device at the same time. However, the parallel threads inside the GPU, the calculation results in video memory, and the DMA transfer queue cannot be atomically captured through the standard virtualization interface. Therefore, virtual machines with vGPU passthrough devices do not support online snapshot functionality.

[0034] (2) A distributed file system refers to a storage architecture that distributes file data across multiple physical storage nodes and achieves unified namespace access and data redundancy through network protocols. A distributed file system typically consists of three components: management nodes, storage nodes, and clients. When a business system writes data, the client writes the data to multiple storage nodes simultaneously to form multiple copies, based on the storage location allocated by the management node, ensuring that data is not lost when a single node fails. This architecture allows the storage capacity and I / O performance of the file system to be linearly expanded by adding storage nodes, while the multi-copy mechanism provides data redundancy protection. In the scheme of this application, the distributed file system carries the virtual disk files of the virtual machine. The management node SDM maintains the metadata and data block distribution information of the virtual disk files, the storage node SDS stores the actual data blocks of the virtual disk and maintains three-copy redundancy, and the client SDC is deployed on the virtualization host to convert the disk I / O of the virtual machine into network read and write to the SDS. This application adds a data change log recording function to the SDC. When the virtual machine writes data, the SDC not only writes the data to the SDS, but also generates a change log containing logical block addresses and operation types and synchronizes it to the SDM. The newly added backup node program, sdb, is deployed outside the cluster. It obtains data change logs by communicating with SDM, and then directly reads the changed data block content from SDS based on the logical address information in the logs to generate incremental backup files. This design leverages the multi-replica feature of the distributed file system, allowing sdb to read data from replica nodes with lower loads, thus avoiding impacting the normal access performance of the business system.

[0035] Please see Figure 1 This is a flowchart illustrating an incremental backup method based on a distributed file system in an embodiment of this application.

[0036] S101. In response to the preset backup trigger condition, determine the set of data change logs within the target time window.

[0037] The preset backup trigger conditions refer to the rules for determining when the backup device starts an incremental backup task. These include time-based triggering based on time periods, event-based triggering based on data change thresholds, and instant triggering based on administrator manual commands. The target time window represents the time range within which data change information needs to be collected for this incremental backup. It typically starts from the timestamp of the previous backup task completion and extends to the timestamp of the current backup task start, forming a continuous time interval. The data change log set represents a structured summary of all data modification operations recorded by the distributed file system within the target time window. This set contains multiple data change log records, each detailing the key attributes of a data write or delete operation. A data change log record is a structured data entry generated by the distributed file system's client component when it captures a write operation performed on a virtual disk file by the business system. This record includes at least the logical address information of the data block where the data change occurred, the operation type identifier, the timestamp of the operation, and the length of the data involved in the operation. Logical address information refers to the location identifier of a data block in the logical address space of the file system. It is usually composed of a file identifier, block offset, and data length, and is used to uniquely locate a specific data block in the file system. It is converted to a physical address on the physical storage medium through a mapping table. Backup devices refer to dedicated backup node servers deployed outside the distributed file system cluster. They run the backup node program sdb, which is responsible for communicating with the management node sdm to obtain data change logs and reading the actual data content from the storage node sds to generate incremental backup files.

[0038] The system monitors whether the preset backup trigger conditions are met through periodic polling or subscription mechanisms. When the timer reaches the preset backup time point, or the cumulative amount of changed data notified by the management node SDM exceeds the preset threshold, or an immediate backup command is received from the administrator through the command line interface, the system determines that the backup trigger conditions are met and starts the incremental backup process.

[0039] When the distributed file system continuously handles data read and write operations for the business system, the client component SDC captures all write and delete operations on the virtual disk files in real time and converts these operations into data change log records, which are then synchronized to the management node SDM and each storage node SDS, forming a distributed change log storage. The timestamp of the last successful backup task is read from local persistent storage as the start boundary of the target time window, and the current system time is obtained as the end boundary of the target time window. These two timestamps determine the time range that the incremental backup needs to cover. Subsequently, a data change log query request is sent to the management node SDM, carrying the start and end timestamps of the target time window in the request parameters. Upon receiving the request, the management node SDM performs a time range filtering operation in its maintained global data change log index, filtering out all data change log records whose operation timestamps fall within the target time window. These records are then sorted in ascending order of timestamps and encapsulated into a data change log set, which is returned to the backup device. After receiving the data change log set, it is cached in memory for subsequent steps to perform frequency statistics and data reading operations. At the same time, the start timestamp of this backup task is persisted to local storage as the basis for determining the starting boundary of the target time window for the next backup task.

[0040] S102. By traversing the data change log set, determine the total number of times the logical address appears as a change object in the data change log set, and determine the total number of times as the update frequency value of the data block corresponding to the logical address.

[0041] In a distributed file system, a logical address is a unique identifier for a data block within its logical address space. It typically consists of a file identifier, file offset, and data block length triple, used to locate a specific data block without relying on its physical storage location. A change object represents the target data block affected by a modification or deletion operation in a data change log record, identified by the logical address information contained in that log record. A data block is the basic unit of data storage and management in a distributed file system. Virtual disk files are divided into fixed-size data blocks for storage. Each data block has an independent logical address and may be distributed across different storage nodes to achieve multi-replica redundancy.

[0042] After successfully obtaining the data change log set within the target time window, a mapping relationship storage structure is established between logical addresses and their occurrence counts to record the cumulative number of times each logical address is recorded in the log set. Then, the traversal process is initiated, sequentially reading each log record in the data change log set according to its timestamp. The logical address information field is extracted from the current log record. This field typically includes a file identifier to indicate the virtual disk file to which the data block belongs, a block offset to indicate the starting position of the data block within the file, and a data length to indicate the byte range involved in this modification operation. The extracted logical address is then queried in the mapping relationship storage structure. If a record already exists for that logical address, it means the data block has appeared in previous log records, and the count value corresponding to that logical address is incremented by one, representing the cumulative modification count of the data block. If no record exists for that logical address, it means this is the first modification of the data block within the target time window, and a new record for that logical address is created in the mapping relationship, with the count value initialized to one. After processing all log records in the data change log set, the mapping relationship storage structure stores the logical addresses of all data blocks that have undergone data changes within the target time window, along with their corresponding occurrence counts. The count value corresponding to each logical address is directly determined as the update frequency value of the data block corresponding to that logical address, and the correspondence between logical addresses and update frequency values ​​is saved.

[0043] S103. Based on the preset frequency threshold, divide the data block into a low-frequency update set and a high-frequency update set.

[0044] After completing the statistics on the update frequency values ​​of all data blocks, the data blocks are classified according to their frequency characteristics to support differentiated backup strategies, thereby reducing the consistency risk caused by data being modified again during the backup process.

[0045] Specifically, the first step is to obtain a preset frequency threshold parameter. This threshold can be a fixed value pre-set in the configuration file by the system administrator based on historical backup experience, such as defining data blocks updated more than ten times as high-frequency updates. Alternatively, it can be an adaptive value dynamically calculated based on the statistical characteristics of the current data change log set. The mapping relationship between logical addresses and update frequency values ​​generated in step S102 is iterated, and the update frequency value of each data block is compared with the preset frequency threshold. When the update frequency value of a data block is less than the preset frequency threshold, the data block is determined to be a low-frequency update type, and its logical address is added to the low-frequency update set. When the update frequency value of a data block is greater than or equal to the preset frequency threshold, the data block is determined to be a high-frequency update type, and its logical address is added to the high-frequency update set. During the partitioning process, the number of data blocks and the total data volume in each set are also recorded to assess the workload distribution of the backup task. For example, if the low-frequency update set contains 80% of the data blocks while the high-frequency update set contains only 20%, it indicates that most of the data is relatively stable, and the consistency risk during the backup process is mainly concentrated on a small number of high-frequency data blocks. After partitioning, the low-frequency update set and the high-frequency update set are stored in independent data structures, and different processing priorities are set for the two sets. The low-frequency update set is marked as a priority processing object because these data blocks are less likely to be modified again during backup reading, and data collection can be completed in the early stage of the backup process. The high-frequency update set is marked as a delayed processing object, and it needs to wait until all low-frequency data blocks are backed up before reading. This shortens the time window between reading and writing high-frequency data blocks to the backup file, and reduces the risk that the data will be modified again by the business system during transmission, resulting in inconsistencies between the backup content and the actual state.

[0046] It is suitable for distributed file system scenarios that host vGPU virtual machines, because GPU-intensive applications frequently modify the virtual disk data blocks corresponding to the video memory mapping area and rendering buffer, while the operating system kernel files and application library files remain relatively stable. By processing these two types of data blocks separately, the internal consistency of backup data can be guaranteed to the greatest extent when snapshot technology cannot be used.

[0047] S104. Determine the first incremental data corresponding to the low-frequency update set based on the logical address information of each data block in the low-frequency update set.

[0048] In this context, a storage node refers to a server node in a distributed file system that actually stores the content of data blocks. Each data block is typically replicated across multiple storage nodes for redundancy protection. A data read request is a network request message sent to a storage node, containing parameters such as the logical address and read length. Upon receiving the request, the storage node reads the corresponding data block content from its local disk and returns it to the backup device via the network.

[0049] After classifying the data blocks by frequency, following the strategy of processing low-frequency data blocks first and then high-frequency data blocks, the actual data content is read from the data blocks in the low-frequency update set. Specifically, all logical addresses in the low-frequency update set are traversed, and the file identifier and block offset information contained in each logical address are parsed. Then, the management node (SDM) of the distributed file system is queried to see which storage nodes (SDS) currently store the data block corresponding to that logical address. The SDM returns a list of all storage nodes storing a copy of that data block based on its maintained data block distribution mapping table. From the returned list, the storage node with the lowest network latency or the lightest load is selected as the target node for this read operation. A data read request message is constructed, containing parameters such as the file identifier of the logical address, the offset within the block, the length of data to be read, and the network address of the backup device. This request is sent over the network to the selected storage node. Upon receiving the request, the storage node locates the physical sector of the data block on its local disk based on the mapping between logical and physical addresses. It then performs a disk read operation to load the data block content into the storage node's memory buffer and encapsulates the data content into a response message, which is transmitted over the network back to the backup device. After receiving the data content returned by the storage node, it is written to a local temporary cache area, and the correspondence between the logical address of the data block and the cache location is recorded for indexing and locating when writing data to the incremental backup file in subsequent steps. The above read operations are performed sequentially according to the logical addresses in the low-frequency update set until all data block content in the low-frequency update set has been successfully read and cached locally. At this point, all data block content in the cache area is aggregated to determine the first incremental data. During data reading, the execution status of each read request is monitored. If a read request times out or returns an error due to network failure or disk anomaly on a storage node, the system automatically switches to another storage node containing a replica of that data block and re-initiates the read request. This leverages the multi-replica feature of the distributed file system to ensure the reliability of data reading. Since the data blocks in the low-frequency update set are updated less frequently within the target time window, the probability of these blocks being modified again by the business system during read operations is relatively low. Therefore, prioritizing the reading of low-frequency data blocks effectively reduces the risk of inconsistencies between backup data and the actual storage state. This is especially true for relatively stable data areas such as virtual machine operating system partitions and application installation directories, where content changes are minimal during backup reading, thus ensuring the consistency and integrity of this incremental data.

[0050] S105. After all the first incremental data has been acquired, determine the second incremental data corresponding to the high-frequency update set based on the logical address information of each data block in the high-frequency update set.

[0051] After completing the read operations of all data blocks in the low-frequency update set and confirming that the first incremental data has been fully cached locally, all logical addresses in the high-frequency update set are traversed. A data read process similar to step S104 is adopted, querying the management node SDM for the storage location of the data block corresponding to each logical address. A suitable target node is selected from the returned list of storage nodes, a data read request is constructed and sent to the storage node, and the data block content returned by the storage node is received and written to the local cache area. The key difference from the low-frequency data block read process is that the high-frequency data block read operations are concentrated in the later stages of the backup task. The time interval between the moment a high-frequency data block is read and the moment its content is written to the incremental backup file is minimized. This timing arrangement reduces the time window during which high-frequency data blocks are modified again by the business system during the backup process. For example, data blocks in a virtual machine's database transaction log file or memory swap partition are continuously written to during virtual machine operation. If these data blocks are read at the start of a backup task, the content of the read data blocks may have been modified multiple times before the backup file is finally generated, since the entire backup process may last for several hours. This results in a significant time difference between the states of high-frequency data blocks and low-frequency data blocks recorded in the backup file. During data recovery, consistency issues may arise, such as missing records pointed to by database indexes or mismatches between memory page tables and actual memory content. By delaying the reading of high-frequency data blocks until after all low-frequency data blocks are read, it ensures that the high-frequency data blocks are read near the end of the backup task. After reading, the operation of writing to the incremental backup file can be performed immediately, thereby bringing the state timestamps of high-frequency and low-frequency data blocks as close as possible, improving the overall consistency level of the backup data. During the reading of the second incremental data, the read timestamp of each high-frequency data block is also recorded. This timestamp information will be used for consistency checks and data correction operations in subsequent steps.

[0052] In some embodiments, to improve the execution efficiency of the high-frequency data block reading phase and reduce read latency caused by storage node performance fluctuations, the storage nodes involved in the high-frequency update set can be pre-evaluated and optimized during the time period of acquiring the first incremental data corresponding to the low-frequency update set. Specifically, while performing the low-frequency data block reading operation in step S104, a parallel monitoring thread is started to specifically perform health status checks on the preferred high-frequency target servers corresponding to each data block in the high-frequency update set. The preferred high-frequency target server refers to the priority read target storage node pre-allocated by the management node SDM for each data block in the high-frequency update set based on the data block replica distribution information and load balancing strategy. Typically, the node with the closest network distance or the fastest historical response speed is selected. The monitoring thread periodically sends probe requests to each preferred high-frequency target server to extract the current network response latency and disk I / O wait queue length of each preferred high-frequency target server. The network response latency is obtained by measuring the round-trip time between sending the probe request and receiving the server response. The disk I / O wait queue length is obtained by querying the disk subsystem statistics of the storage node operating system. This indicator reflects the number of disk read / write requests currently waiting to be processed by the storage node. Based on the current network response latency and disk I / O wait queue length, the node response latency multiplier for each preferred high-frequency target server is calculated. The node response latency multiplier is a comprehensive evaluation metric. It is calculated by comparing the current network response latency with the server's historical average response latency to obtain the latency increase ratio, and by comparing the current disk I / O wait queue length with a preset health threshold to obtain the overload ratio. These two ratios are then weighted and summed to obtain the node response latency multiplier value. A higher value indicates a more severe performance degradation compared to normal levels. When the node response latency multiplier of a preferred high-frequency target server exceeds a preset performance degradation threshold, it is determined that the server may not provide ideal read performance during subsequent high-frequency data read phases, and an alternative node needs to be found in advance. Based on the target logical address corresponding to the preferred high-frequency target server that generated the node response latency multiplier, alternative node addressing is performed to determine the alternative server to replace the preferred high-frequency target server. By using this pre-evaluation and node replacement mechanism, which is executed in parallel during the low-frequency data reading phase, the optimized scheduling of storage nodes can be completed before the high-frequency data reading officially begins. This avoids reading delays caused by node performance issues at critical moments of high-frequency data reading, thereby further shortening the time window between reading high-frequency data blocks and writing them to backup files.

[0053] In some embodiments, a security filtering method based on replica synchronization timestamps is employed. When the monitoring thread detects that a preferred high-frequency target server is generating an excessive node response latency multiplier, it records the target logical address corresponding to that server and then sends a metadata query request to the management node (SDM). The request parameters carry the target logical address. The management node uses the metadata dictionary to look up the distribution information of all replicas of the data block corresponding to that logical address. The metadata dictionary is a mapping table structure maintained by the management node, recording on which storage nodes each logical address's data block has a replica and the synchronization status information of each replica. The management node returns a list of all secondary candidate slave servers storing replicas of the data block corresponding to the target logical address to the backup device. Secondary candidate slave servers refer to other storage nodes besides the preferred high-frequency target server that store replicas of that data block. After receiving the list of secondary candidate slave servers, a replica status query request is sent to each server in the list to obtain the latest replica synchronization timestamp of each secondary candidate slave server. The replica synchronization timestamp records the time when the data block replica on that server was last synchronized and updated from the primary replica node. Simultaneously, a health history query request is sent to the primary high-frequency target server to obtain its last health timestamp. The last health timestamp refers to the last time the server was in a normal state before performance indicators began to deteriorate. This is determined by analyzing the server's historical monitoring data, specifically the moment when the node response latency ratio first exceeded a preset threshold. The replica synchronization timestamps of each secondary candidate slave server are compared with the last health timestamp of the primary high-frequency target server. Secondary candidate slave servers with replica synchronization timestamps later than or equal to the last health timestamp are grouped into a secure backup cluster. The data block replica content on these servers is consistent with or updated to the replica content of the primary high-frequency target server when it was in a healthy state, ensuring that the read data is not outdated due to replica synchronization delays. Then, the secondary candidate slave server with the highest historical average transfer rate is selected from the secure backup cluster as the backup server. The historical average transfer rate is obtained by querying a locally maintained storage node performance statistics database, which records the actual data transfer rates during multiple backup tasks with each storage node. The arithmetic mean of the historical transfer rates for each node is calculated as the evaluation metric. The read request node corresponding to the target logical address is updated to the selected alternative server. When high-frequency data reading is performed subsequently, data read requests involving the target logical address will be sent to the alternative server instead of the original preferred high-frequency target server, thereby avoiding nodes with degraded performance.

[0054] S106. Write the first incremental data and the second incremental data into the incremental backup file.

[0055] An incremental backup file is a file object created by the backup device on a local storage system or a remote backup storage device to persistently store incremental data. This file contains data block content, metadata information, and an index structure, used to reconstruct the virtual disk file's state at a specific point in time during data recovery. A write operation refers to the disk I / O process of persisting the first and second incremental data cached in memory to the incremental backup file according to a specific format and order.

[0056] After the backup device completes the reading operations of the first and second incremental data and caches all data block contents in local memory, the incremental backup file writing process begins. A new incremental backup file is created in the local file system or remote backup storage system. The file name typically includes the timestamp of the backup task and the identification information of the source virtual disk file. Metadata information of the backup task is written to the header area of ​​the incremental backup file. Then, a data block index structure is built, traversing all data blocks in the first and second incremental data, generating an index record for each data block. The index record contains the logical address of the data block, the physical offset of the data block in the incremental backup file, the length of the data block, and the checksum of the data block content. All index records are arranged in logical address order and written to the index area of ​​the incremental backup file. After the index area is written, the data block content area is written. Following the order of writing the first incremental data first and then the second incremental data, the data block contents in the memory cache are appended to the incremental backup file sequentially. After each data block is written, the physical offset field of that data block in the index record is updated to ensure that the index information is consistent with the actual data location.

[0057] In some embodiments, to further ensure the accuracy and consistency of the second incremental data written to the incremental backup file, after the first and second incremental data are written to the incremental backup file, a consistency verification process for the second incremental data also needs to be performed. This is especially important when the second incremental data is read from a backup server rather than the primary high-frequency target server; it is necessary to confirm whether the replica data on the backup server is fully synchronized with the primary node at the time of reading. Specifically, the delivery timestamp of the service response message sent by the backup server to the backup device when transmitting the second incremental data is obtained. The delivery timestamp is recorded in the timestamp field of the service response message, indicating the precise time when the backup server completes the data block reading and encapsulates the data content into a response message before sending it to the network. All target logical addresses from which data was read from the backup server in step S105 are extracted. For each target logical address, an out-of-band alignment query request is initiated to the master control component of the distributed cluster based on the delivery timestamp corresponding to that logical address. The master control component refers to the core management module in the distributed file system responsible for coordinating data consistency and replica synchronization, typically deployed on the management node SDM. Out-of-band alignment query requests are a dedicated query mechanism for verifying replica consistency. The request parameters include the target logical address, the replica synchronization timestamp, and the delivery timestamp of the backup server. Upon receiving the request, the master control component searches its maintained global change log for any data change operations performed by the master node but not yet distributed to the backup server within the time interval between the replica synchronization timestamp and the delivery timestamp for the target logical address. The master control component encapsulates the search results into a change log entry for which the master node has not distributed the data and returns it to the backup device. If the log is empty, it means that the backup server's replica was fully synchronized with the master node when delivering the data, and the data read is the latest version. If the log is not empty, it means that during the time period when the backup server reads the data and sends a response, the master node performed new modifications on the data block, but these modifications have not yet been synchronized to the backup server, resulting in slightly outdated replica data being read. Upon receiving feedback from the master control component that the master node has not issued a change log, the system checks if the log is empty. If the log is not empty, it determines that there are unsynchronized data segments in the second incremental data. Unsynchronized data segments refer to data blocks where the data read from the backup server differs from the current actual data content of the master node due to replica synchronization delays. After identifying the unsynchronized data segments, a data correction strategy is triggered. This involves updating the outdated data written to the incremental backup file by rereading the latest data from the master node or applying the incremental modifications from the change log. This ensures that all data blocks in the backup file reflect the true state of the master node at the time of backup, preventing data version inconsistencies caused by replica synchronization delays from affecting subsequent data recovery operations.

[0058] In some embodiments, a data correction strategy for unsynchronized data segments can be implemented in various ways. Optionally, the backup device adopts an incremental repair method based on in-situ overwrite. When it is determined that there are unsynchronized data segments in the second incremental data, the physical offset range corresponding to the second incremental data in the incremental backup file is extracted as the data area to be repaired. The physical offset range is obtained by querying the index information recorded by the backup device when writing data in step S106. The index information stores the starting offset and data length of each data block in the incremental backup file. The boundary of the data area to be repaired can be determined by summing the offset ranges of all high-frequency data blocks read from the alternative server. Then, based on the master node's unissued change log returned by the master control component, each change operation recorded in the log is parsed to extract the missing data fragments that were not synchronized to the backup server within the replica synchronization timestamp and delivery timestamp interval, along with the corresponding intra-block offset addresses. The missing data fragment refers to the actual data content involved in the write operations performed by the master node on the data block within that time interval. The intra-block offset address refers to the starting position and length range of these write operations within the data block. For example, if a change log record shows that the master node wrote 512 bytes of new data at the 1024th byte position of the data block, then the missing data fragment is these 512 bytes, and the intra-block offset address is the starting position of the 1024th byte and the length of the 512 bytes. The content of the data area to be repaired in the incremental backup file is read into the local cache memory. Within the backup device's cache, the corresponding old data stream is located within the data area to be repaired according to the intra-block offset address. The old data stream refers to the portion of the outdated replica data read from the backup server that corresponds to the missing data fragment position. The missing data fragments are used to overwrite the old data stream. The memory location of the old data stream in the cache is replaced byte-by-byte with the content of the missing data fragments. If a data block has multiple change log records, each change operation is applied sequentially according to the log timestamp order, ensuring that the final data state is completely consistent with the state of the master node at the delivery timestamp. After overwriting all missing data fragments, a block checksum is calculated based on the data area to be repaired after the overwrite. The block checksum is generated by performing a hash algorithm such as SHA-256 on the corrected data block content and is used to verify the correctness of the data correction operation. The corrected data area to be repaired is rewritten to the corresponding physical offset position in the incremental backup file, overwriting the original outdated data. Simultaneously, the checksum field of the data block in the incremental backup file index area is updated, replacing the original checksum calculated based on the outdated data with the newly calculated block checksum. The block checksum is used to update the integrity verification information of the incremental backup file. During data recovery, the integrity and accuracy of the backup data can be verified by recalculating the hash value of the data block and comparing it with the checksum recorded in the index.

[0059] In other embodiments, when an unsynchronized data segment is detected in the second incremental data, a data reread request is initiated to the master node to obtain the latest complete data block content. Specifically, the master node's unissued change logs are parsed to extract a list of all target logical addresses involved in data changes. For each target logical address in the list, the address of the primary replica node corresponding to the data block is queried from the management node SDM. The primary replica is the authoritative replica in the distributed file system responsible for handling write operations and synchronizing data with other replica nodes; its data status is always up-to-date. A data read request is constructed for the primary replica node, with the request parameters including the target logical address and a high-priority flag. The high-priority flag is used to notify the primary replica node that the read request is a backup consistency correction operation that needs to be processed first. After receiving the request, the primary replica node reads the latest data block content corresponding to the logical address from its local storage and returns it to the backup device. After receiving the latest data block content returned by the primary replica node, the physical offset range corresponding to the data block in the incremental backup file is extracted. The old data block content at the offset position in the incremental backup file is completely replaced with the latest data block content read from the primary replica node. The replacement operation is implemented by locating the target offset through the file random access interface and then performing a write operation. A new checksum is calculated for the replaced data block content. The checksum field and data source identifier field of that data block in the incremental backup file index area are updated. The data source identifier field records that the data block was corrected by rereading from the primary replica node, facilitating subsequent auditing and problem tracing. After all data blocks with replica asynchronous lag issues have completed primary node rereading and replacement operations, the global checksum of the incremental backup file is recalculated. The new global checksum is written to the end of the backup file, overwriting the original global checksum calculated based on the file content containing outdated data. It is understandable that other methods can also be used to correct unsynchronized data fragments, such as a hybrid correction strategy. Incremental patching can be used for data blocks with small changes, while full replacement can be used for data blocks with changes exceeding a certain percentage threshold of the data block size. The optimal correction method can be dynamically selected based on the actual changes to balance correction efficiency and network overhead; this is not limited here.

[0060] In this embodiment, an incremental backup method based on data change logs is adopted. By recording data block-level change operations in the client component of the distributed file system and synchronizing the logs to the management node, it is possible to identify data blocks that have changed within the target time window without relying on the snapshot function of the virtualization platform. This enables online incremental backup capabilities for virtual machines with vGPU passthrough devices. By statistically analyzing the update frequency of each data block within the target time window and dividing the data blocks into low-frequency update sets and high-frequency update sets according to a preset frequency threshold, it is possible to identify data blocks with a high risk of further modification during the backup process. By adopting a phased strategy of backing up low-frequency data blocks first and then high-frequency data blocks, the time window between reading and writing high-frequency data blocks to the backup file is minimized, effectively reducing the consistency risk caused by the backup data being modified again by the business system during transmission.

[0061] In practical applications, some data blocks, despite having low update frequency, may have strong relationships with other high-frequency data blocks (such as database index files and data files, or file system inodes and data blocks). These related data blocks need to be kept synchronized for backup to ensure data integrity after recovery. Assigning related data blocks to different sets solely based on update frequency may lead to time consistency issues in the backup data.

[0062] Please see Figure 2 This is another flowchart illustrating an incremental backup method based on a distributed file system in an embodiment of this application.

[0063] S201. In response to the preset backup trigger condition, determine the set of data change logs within the target time window.

[0064] S202. By traversing the data change log set, determine the total number of times the logical address appears as a change object in the data change log set, and determine the total number of times as the update frequency value of the data block corresponding to the logical address.

[0065] Steps S201~S202 and Figure 1 In the illustrated embodiment, steps S101 to S102 are similar, and can be found in the description of steps S101 to S102 in section 1. They will not be repeated here.

[0066] S203. Based on the file system metadata corresponding to the data block, obtain the number of hard links associated with other directory entries for each data block's inode.

[0067] File system metadata refers to structured data in a distributed file system that describes management information such as file and data block organization, access permissions, timestamps, and storage locations. This metadata is typically stored in a metadata database maintained by the management node (SDM), separate from the actual data block content. An inode is a data structure in a file system used to uniquely identify a file or directory object. In Unix-like file systems, it contains attributes such as file owner information, access permissions, creation time, modification time, file size, and a list of pointers to the actual data blocks, but not filename information. Hard links are a mechanism in a file system that allows multiple different directory entries to point to the same inode. When a hard link is created, the actual data content of the file is not copied; instead, a new directory entry is added to the directory pointing to the existing inode. The inode internally maintains a hard link counter to record how many directory entries reference the node.

[0068] After the backup device completes the statistics of update frequency values ​​for all data blocks, it extracts a list of logical addresses of all data blocks to be backed up from the mapping relationship between logical addresses and update frequency values ​​generated in step S202. Then, it initiates a batch metadata query request to the management node (SDM) of the distributed file system. The SDM retrieves the inode identifier of the data block corresponding to each logical address from its maintained metadata database, and further queries the inode table to obtain the hard link count field value for each inode. The SDM organizes the hard link count of the inode corresponding to each logical address into structured response data and returns it to the backup device. It stores the correspondence between logical addresses and hard link counts in its local data block attribute mapping table, providing basic data for calculating the concurrency association weight coefficient in subsequent steps. By analyzing the distribution characteristics of hard link counts, key shared resources in the file system can be identified, and it can be determined whether there are logical dependencies between data blocks.

[0069] S204. If the arithmetic mean of the number of hard links is greater than the preset dependency threshold, calculate the probability that each data block will be concurrently written to disk with other data blocks in the same transaction log within the preset historical period, and use this probability as the concurrency aggregation degree.

[0070] Transaction logs refer to log files in a distributed file system that record the atomic execution process of data write operations. When modification operations on multiple data blocks are organized and committed within the same transaction, the write records of these data blocks will appear in the same transaction log, indicating that these data blocks are logically related. Concurrent disk writes refer to the behavior of committing write operations on multiple data blocks to persistent storage media within the same transaction. Data blocks concurrently written to disk typically belong to different components of the same business operation at the application level.

[0071] After obtaining the number of hard links for all data blocks, calculate the arithmetic mean of these hard link counts, summing the hard link counts of all data blocks and dividing by the total number of data blocks. Compare the calculated arithmetic mean with a preset dependency threshold. The preset dependency threshold is typically set based on the file system type and application scenario; for example, it might be set to 1.5 for database storage and 1.2 for ordinary file storage. If the arithmetic mean of hard link counts is greater than the preset dependency threshold, it indicates that there are many shared files or symbolic links in the current data block set, suggesting potentially complex logical dependencies between data blocks. Further analysis of the concurrent modification patterns of the data blocks is needed to accurately assess the associated risks. Request the transaction log archive data within a preset historical period from the management node SDM. The start time of the preset historical period is typically set to the current time minus the historical period length. The management node retrieves all transaction log records within this time range from the transaction log storage system. Each transaction log record contains a transaction identifier, a transaction commit timestamp, and a list of logical addresses of all data blocks involved in the transaction. After receiving historical transaction log data, a concurrency aggregation degree calculation is performed on each data block that needs to be backed up. Specifically, all transaction log records are traversed. For each transaction log, if the list of data blocks involved in the transaction contains the logical address of the target data block, the transaction is recorded as a modification event of the target data block. Simultaneously, the logical addresses of other data blocks in the transaction besides the target data block are recorded as associated data blocks for concurrent modification. The total number of transaction logs in which the target data block appears within the historical period is counted, recorded as the total number of modification transactions for that data block. Then, the number of transactions containing at least one other data block within these modification transactions is counted, recorded as the number of concurrent modification transactions for that data block. Dividing the number of concurrent modification transactions by the total number of modification transactions yields the probability that the data block and other data blocks concurrently appear in the same transaction log and are written to disk. This probability value is the concurrency aggregation degree of that data block. After completing the concurrency aggregation degree calculation for all data blocks, the correspondence between logical addresses and concurrency aggregation degrees is stored in a data block attribute mapping table, providing key parameters for subsequent steps to calculate the concurrency association weight coefficient. By analyzing the concurrent modification patterns of data blocks, we can identify groups of data blocks that have strong relationships at the business logic level. These data blocks usually correspond to database index files and data files, configuration files and log files, and other file pairs that need to be kept in sync. During the backup process, they should be treated as a whole to avoid consistency issues.

[0072] S205. After normalizing the number of hard links and the degree of concurrent aggregation, the concurrent association weight coefficient, which represents the risk of each data block being associated and modified, is obtained by weighting the data with preset weight parameters.

[0073] After obtaining the hard link count and concurrency aggregation statistics for all data blocks, the hard link count is normalized. The minimum and maximum hard link counts for all data blocks are found. For each data block, the minimum hard link count is subtracted, and then divided by the difference between the minimum and maximum values ​​to obtain the normalized hard link count index. This index is mapped to a range of zero to one. The normalized value for the data block with the minimum hard link count is zero, the normalized value for the data block with the maximum hard link count is one, and the normalized values ​​for other data blocks are proportionally distributed between zero and one. Then, the concurrency aggregation is normalized in the same way. Since the concurrency aggregation is itself a probability value and its original value range is already between zero and one, a minimum-maximum normalization operation is still performed to maintain consistency with the hard link count normalization method. The minimum and maximum concurrency aggregation values ​​for all data blocks are found, and the minimum value is subtracted from the range for the concurrency aggregation value of each data block to obtain the normalized concurrency aggregation index. After normalizing the two metrics, the preset weight parameters are read from the configuration file. These weight parameters include two values: hard link weight and concurrent aggregation weight. A weighted calculation is performed on each data block: the normalized number of hard links is multiplied by the hard link weight, and the normalized concurrent aggregation degree is multiplied by the concurrent aggregation weight. The two products are then added together to obtain the concurrent association weight coefficient for that data block. After calculating the concurrent association weight coefficients for all data blocks, the correspondence between logical addresses and concurrent association weight coefficients is stored in the data block attribute mapping table.

[0074] S206. By sorting the sequence of concurrent association weight coefficients corresponding to all data blocks according to their numerical values, the boundary point value with the largest difference between adjacent weight coefficients is identified, and the boundary point value is used as the updated preset frequency threshold.

[0075] The adjacent weight coefficient difference refers to the absolute value of the numerical difference between two adjacent elements in the sorted sequence. This difference reflects the magnitude of the numerical jump in the sequence at that position. The dividing point value refers to the weight coefficient value at the sequence position corresponding to the maximum value of the adjacent weight coefficient difference. This value divides the entire data block into two subsets with significantly different correlation characteristics.

[0076] After calculating the concurrent association weight coefficients for all data blocks, the original frequency threshold settings need to be optimized based on these weight coefficients. This ensures that the classification of data blocks considers not only the update frequency itself but also the strength of the association between data blocks. Specifically, the concurrent association weight coefficients corresponding to all data blocks are extracted from the data block attribute mapping table. These weight coefficients are organized into a numerical sequence, the length of which is equal to the total number of data blocks to be backed up. The sequence is then sorted in ascending order. The first element in the sorted sequence is the weight value corresponding to the data block with the smallest concurrent association weight coefficient, the last element is the weight value corresponding to the data block with the largest concurrent association weight coefficient, and the intermediate elements are arranged in ascending order of weight coefficient. After sorting, the sorted sequence is traversed, and the numerical difference between each pair of adjacent elements is calculated. For the i-th element and the next i-th element in the sequence, the difference between the next i-th element and the i-th element is calculated, and all adjacent differences are stored in a difference array. The maximum value is found in the difference array, and the sequence position index corresponding to the maximum difference is recorded. Assuming the maximum difference occurs between the k-th element and the next k-th element, the dividing point is index k. The weight coefficient value at index k in the sorted sequence is extracted as the dividing point value. This dividing point value naturally divides the data block set into two subsets. Data blocks with an index less than or equal to k have a lower concurrent association weight coefficient, indicating that these data blocks have a weaker association with other data blocks and are less likely to be modified along with them during backup. Data blocks with an index greater than k have a higher concurrent association weight coefficient, indicating that these data blocks have a stronger association with other data blocks and are more likely to change due to modifications of related data blocks during backup. The identified dividing point value is used as the updated preset frequency threshold, replacing the fixed frequency threshold value set in the original configuration file. In the subsequent step S207, this updated threshold is used to classify the data blocks.

[0077] Understandably, by dynamically determining the frequency threshold based on the natural boundary point of the concurrent association weight coefficient sequence, the classification criteria can be adaptively adjusted according to the actual association characteristics of the current data block set. This avoids the problem of unreasonable classification that may be caused by using a fixed threshold. For example, when data blocks generally have strong associations, a fixed threshold may classify too many data blocks into the high-frequency set, resulting in an excessive amount of data that is delayed in reading. On the other hand, a dynamic threshold can find the optimal classification boundary according to the actual situation of the weight coefficient distribution, making the ratio of low-frequency set to high-frequency set more reasonable, thereby improving the overall execution efficiency of the backup task and the level of data consistency assurance.

[0078] In some embodiments, the identification of the boundary point with the largest difference between adjacent weight coefficients can be achieved in several ways. Optionally, the backup device adopts a single boundary point identification method based on the global maximum difference. After sorting the concurrent correlation weight coefficient sequence in ascending order, the backup device traverses all adjacent element pairs to calculate the difference, finds the unique maximum value among all differences, and uses the position corresponding to the maximum difference as the boundary point. This method assumes that the data block set has obvious polarization in concurrent correlation characteristics, and can clearly divide the data blocks into low correlation groups and high correlation groups through a single boundary point. It is simple to implement and computationally efficient, and is suitable for scenarios where the data block correlation characteristics distribution presents a bimodal pattern. Optionally, the backup device adopts a multi-candidate boundary point identification method based on local maxima. After calculating the adjacent difference sequence, the backup device does not directly select the global maximum difference, but identifies all local maxima points in the difference sequence. Local maxima points are positions where the difference is greater than the differences before and after them. These local maxima points are potential candidate boundary point positions. The backup device calculates a classification quality assessment index for each candidate boundary point. For example, it calculates the sum of the variances of the weight coefficients within the two subsets after dividing at that boundary point. A smaller sum of variances indicates more similar data block association features within the subsets, resulting in higher classification quality. The backup device selects the candidate boundary point with the optimal classification quality assessment index as the final boundary point value. This method can handle complex scenarios where the distribution of data block association features exhibits a multi-peak pattern. By evaluating the classification performance of multiple candidate positions, the optimal boundary point is selected, improving the accuracy and robustness of the classification results. It is understandable that other methods can also be used to identify boundary points, such as an automatic grouping method based on clustering algorithms. This involves using the concurrent association weight coefficients as feature values ​​to perform K-means clustering on the data blocks, setting the number of clusters to two. The clustering result naturally divides the data blocks into two groups, and the midpoint between the cluster centers of the two groups is used as the boundary point value. This is not a limitation here.

[0079] S207. According to the preset frequency threshold, data blocks with update frequency values ​​less than the preset frequency threshold are assigned to the low-frequency update set, and data blocks with update frequency values ​​greater than or equal to the preset frequency threshold are assigned to the high-frequency update set.

[0080] After the backup device determines the updated preset frequency threshold in step S206, it reads the logical address and corresponding update frequency value of each data block from the data block attribute mapping table, and simultaneously obtains the updated preset frequency threshold value calculated in step S206. The backup device traverses all data blocks. For each data block, it compares its update frequency value with the preset frequency threshold. If the update frequency value of the data block is less than the preset frequency threshold, the backup device adds the logical address of the data block to the logical address list of the low-frequency update set, and simultaneously marks the data block as belonging to the low-frequency set in the data block attribute mapping table. If the update frequency value of the data block is greater than or equal to the preset frequency threshold, the backup device adds the logical address of the data block to the logical address list of the high-frequency update set, and simultaneously marks the data block as belonging to the high-frequency set in the data block attribute mapping table. The logical address lists of the low-frequency update set and the high-frequency update set are stored in the execution plan data structure of the backup task, respectively. The logical address list of the low-frequency update set will be used to determine the first incremental data in step S104, and the logical address list of the high-frequency update set will be used to determine the second incremental data in step S105.

[0081] S208. If the arithmetic mean of the number of hard links is less than or equal to a preset dependency threshold, retrieve the physical block mapping table of the logical address corresponding to each data block on the physical storage medium.

[0082] After the backup device calculates the arithmetic mean of the number of hard links in step S203, it compares the average value with a preset dependency threshold. If the arithmetic mean of the number of hard links is less than or equal to the preset dependency threshold, it indicates that most data blocks in the current data block set do not have multi-path reference relationships and the logical association between data blocks at the file system level is low. At this time, the analysis path based on concurrency aggregation degree and concurrency association weight coefficient in steps S204 to S207 is no longer executed, and the backup device adopts an alternative analysis strategy based on physical storage layout characteristics.

[0083] Specifically, the backup device extracts a list of logical addresses of all data blocks to be backed up from the data block attribute mapping table and uses this list as a query parameter to initiate a physical block mapping table query request to the storage nodes of the distributed file system. Upon receiving the query request, the storage node retrieves the physical storage location information corresponding to each logical address from its locally maintained address mapping database. For storage nodes using traditional mechanical hard drives, the physical location information includes the disk number, cylinder number, head number, and sector number; for storage nodes using solid-state drives (SSDs), the physical location information includes the flash memory chip number, block number, and page number; for storage nodes using object storage, the physical location information includes the object bucket identifier and object key value. The storage node organizes the physical location information corresponding to all logical addresses into a structured physical block mapping table and returns it to the backup device. Each record in the mapping table contains fields such as logical address, physical device identifier, physical address coordinates, and data block size. After receiving the physical block mapping table, the backup device stores the mapping information in the data block attribute mapping table, associating each logical address with its corresponding physical storage location information. By retrieving the physical block mapping table, the backup device can understand the actual distribution of data blocks on the physical storage medium.

[0084] S209. Calculate the spatial offset of adjacent data blocks at each logical address on the physical storage medium, and define the distribution ratio of data blocks with spatial offsets greater than a preset continuity threshold as spatial sparsity.

[0085] In a distributed file system, physical storage media refers to the hardware devices that actually store data, including hard disk drives (HDDs), solid-state drives (SSDs), or other non-volatile storage devices. These devices organize storage space using physical sectors or flash pages. A physical block mapping table is a data structure that records the mapping relationship between logical addresses and actual storage locations on the physical storage medium. This mapping table is typically maintained by the file system or block device driver of the storage node and is used to translate logical addresses at the file system level into physical sector addresses or flash page addresses at the storage device level. A logical address is an abstract address identifier used to identify data blocks in a distributed file system. It is independent of the physical storage location, providing location transparency and flexible data migration capabilities. A physical block is the smallest addressable storage unit on the physical storage medium, typically a 512-byte or 4096-byte sector for hard disk drives and a 4KB or 8KB flash page for solid-state drives.

[0086] The backup device sorts all data blocks to be backed up in ascending order of logical address, resulting in a logically ordered sequence of data blocks. Data blocks at adjacent positions in this sequence are also logically adjacent in the address space. The backup device traverses this ordered sequence, and for each pair of adjacent data blocks, it extracts the physical storage location information corresponding to these two data blocks from the physical block mapping table and calculates the spatial offset between the two physical locations. For mechanical hard disk storage, the backup device first determines whether the two data blocks are located on the same physical disk. If they are on different disks, the spatial offset is set to infinity, indicating complete discontinuity. If they are on the same disk, the difference in physical sector numbers between the two data blocks is calculated as the spatial offset. This sector number difference equals the starting sector number of the second data block minus the ending sector number of the first data block. For solid-state drive storage, the backup device calculates the difference in physical block numbers between the two data blocks as the spatial offset. If the two data blocks are located on different flash memory chips, an inter-chip jump penalty coefficient is added to the block number difference. The backup device compares the calculated spatial offset with a preset continuity threshold. If the spatial offset is greater than the threshold, the backup device marks the data block pair as physically discontinuous and increments the discontinuous pair counter. If the spatial offset is less than or equal to the threshold, the backup device marks the data block pair as physically continuous. After calculating the spatial offset and determining the continuity of all adjacent data block pairs, the backup device counts the total number of physically discontinuous pairs and divides this number by the total number of adjacent data block pairs to obtain the spatial sparsity. The calculated spatial sparsity is stored as an overall attribute of the data block set in the backup task's metadata.

[0087] Understandably, by analyzing the spatial dispersion sparsity of data blocks, backup devices can assess the physical storage layout quality of the current data block set. When the spatial dispersion sparsity is high, it means that the data blocks are highly dispersed on the physical medium. If these data blocks are read sequentially according to their logical address order, it will lead to a large number of random disk access operations, affecting read performance.

[0088] S210. Perform a weighted operation on the update frequency value and the spatial discrete sparsity to obtain the addressing index corresponding to each data block.

[0089] The backup device reads the update frequency value of each data block from the data block attribute mapping table, and simultaneously obtains the overall spatial dispersion sparsity value calculated in step S209. Since spatial dispersion sparsity is an overall attribute of the data block set rather than an attribute of a single data block, the backup device needs to convert this overall attribute into an individual attribute of each data block, and adjust the overall spatial dispersion sparsity based on the local continuity characteristics of each data block on the physical storage medium. For each data block, the physical spatial offset between the data block and its logically adjacent preceding and following data blocks is checked. If both offsets are less than a preset continuity threshold, the data block is in a physically continuous region, and its local dispersion is set to zero. If either offset is greater than the preset continuity threshold, the local dispersion is set to half of the overall spatial dispersion sparsity. If both offsets are greater than the preset continuity threshold, the local dispersion is set to twice the overall spatial dispersion sparsity. After calculating the local dispersion of each data block, the backup device performs normalization on the update frequency and local dispersion, mapping both metrics to a value range of zero to one. The normalization method uses minimum-maximum normalization. For the update frequency, the minimum and maximum update frequencies across all data blocks are found. The normalized update frequency is obtained by subtracting the minimum from the update frequency of each data block and then dividing by the range. For local dispersion, the minimum and maximum local dispersion values ​​across all data blocks are found, and the same normalization operation is performed to obtain the normalized local dispersion. The backup device reads preset weighting coefficients from the configuration file. These weighting coefficients include two values: a frequency weight and a dispersion weight. The sum of these two weights equals one. For example, a frequency weight set to 0.6 and a dispersion weight set to 0.4 indicates that the update frequency is slightly more important than physical layout features in the overall evaluation. The backup device performs a weighted operation on each data block, multiplying the normalized update frequency of the data block by a frequency weight and the normalized local dispersion by a dispersion weight, and then adding the two products to obtain the addressing index of the data block. After calculating the addressing index for all data blocks, the backup device stores the correspondence between logical addresses and addressing indices in a data block attribute mapping table, providing basic data for identifying auxiliary thresholds in step S211.

[0090] S211. The boundary value at which the difference between adjacent indices in the sequence of addressing indices reaches the maximum value is established as the auxiliary threshold. Data blocks with addressing indices less than the auxiliary threshold are combined into a low-frequency update set, and data blocks with addressing indices greater than or equal to the auxiliary threshold are combined into a high-frequency update set.

[0091] The backup device extracts the addressing indices corresponding to all data blocks from the data block attribute mapping table and organizes these indices into a numerical sequence. The length of the sequence is equal to the total number of data blocks to be backed up. The backup device performs an ascending sort operation on this sequence. The first element of the sorted sequence is the index value corresponding to the data block with the smallest addressing index, the last element is the index value corresponding to the data block with the largest addressing index, and the middle elements are arranged in ascending order of addressing index. After sorting, the backup device traverses the sorted sequence and calculates the numerical difference between each pair of adjacent elements. For the i-th element and the (i+1)-th element in the sequence, it calculates the difference between the (i+1)-th element and the i-th element, and stores all adjacent differences in a difference array. The backup device finds the maximum value in the difference array and records the sequence position index corresponding to the maximum difference. Assuming that the maximum difference occurs between the k-th element and the (k+1)-th element, the boundary position is index k. The backup device extracts the addressing index value at index k in the sorted sequence as a boundary value, establishing this boundary value as an auxiliary threshold. This auxiliary threshold naturally divides the data block set into two subsets: data blocks with an index less than or equal to k have a lower addressing index, indicating that these data blocks have a higher priority for backup processing after comprehensive evaluation and can be read early in the backup task; data blocks with an index greater than k have a higher addressing index, indicating that these data blocks need to be processed with a delay. Iterates through all data blocks. For each data block, its addressing index is compared with the auxiliary threshold. If the addressing index of the data block is less than the auxiliary threshold, the logical address of the data block is added to the logical address list of the low-frequency update set, and the data block is marked as belonging to the low-frequency set in the data block attribute mapping table. If the addressing index of the data block is greater than or equal to the auxiliary threshold, the logical address of the data block is added to the logical address list of the high-frequency update set, and the data block is marked as belonging to the high-frequency set in the data block attribute mapping table. After classifying all data blocks, the number of data blocks contained in the low-frequency update set and the high-frequency update set are counted, and the logical address lists of the two sets are stored in the execution plan data structure of the backup task. By dynamically determining the auxiliary threshold based on the natural boundary point of the addressing index sequence, the classification criteria can be adaptively adjusted according to the comprehensive distribution of the update frequency and physical layout characteristics of data blocks in scenarios with weak logical relationships between data blocks. This avoids the classification problems that may be caused by using a fixed threshold, making the division of the low-frequency set and the high-frequency set more consistent with the actual characteristics of the current data block set, ensuring the consistency of backup data and optimizing the execution efficiency of the backup task.

[0092] S212. Determine the first incremental data corresponding to the low-frequency update set based on the logical address information of each data block in the low-frequency update set.

[0093] Steps S212~S214 and Figure 1 In the illustrated embodiment, steps S103 to S105 are similar, and can be found in the description of steps S103 to S105 in section 1. They will not be repeated here.

[0094] S213. After all the first incremental data has been acquired, determine the second incremental data corresponding to the high-frequency update set based on the logical address information of each data block in the high-frequency update set.

[0095] S214. Write the first incremental data and the second incremental data into the incremental backup file.

[0096] In this embodiment, the technical feature of dynamically selecting differentiated frequency thresholds to determine the path is adopted based on the comparison result of the arithmetic mean of the number of hard links and a preset dependency threshold. When the arithmetic mean of the number of hard links is greater than the preset dependency threshold, the concurrent aggregation degree of the statistical blocks in the historical period is statistically analyzed, and the concurrent aggregation degree and the number of hard links are normalized and weighted to obtain the concurrent association weight coefficient. Then, the frequency threshold is dynamically updated based on the maximum boundary point of the adjacent difference of the concurrent association weight coefficient sequence. When the arithmetic mean of the number of hard links is less than or equal to the preset dependency threshold, the spatial dispersion sparsity of adjacent data blocks of logical addresses is calculated by retrieving the physical block mapping table, and the spatial dispersion sparsity is weighted with the update frequency value to obtain the addressing index. Then, the auxiliary threshold is established based on the maximum boundary value of the adjacent difference of the addressing index sequence. In two scenarios where the logical correlation between data blocks in the file system differs significantly, the classification boundary between low-frequency update sets and high-frequency update sets is adaptively determined based on the concurrent modification correlation strength characteristics or the discrete characteristics of physical storage layout between data blocks. This avoids the situation where some strongly correlated data blocks or high addressing overhead data blocks are incorrectly classified into the low-frequency update set and read first in the early stages of the backup task due to the failure to perceive the data block correlation structure or physical layout status when using a fixed frequency threshold. As a result, the division between low-frequency update sets and high-frequency update sets more accurately reflects the actual risk of content changes in each data block during the backup reading process, and reduces the internal consistency risks of backup data introduced by high-risk data blocks being read prematurely due to inaccurate data block classification.

[0097] The following describes an exemplary backup device 300 provided in an embodiment of this application. Figure 3 This is an exemplary hardware structure diagram of the backup device 300 provided in this application embodiment.

[0098] In some embodiments, the backup device 300 is a computer device or includes a computer device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements the methods in the embodiments of this application.

[0099] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0100] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0101] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0102] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0103] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for incremental backup based on a distributed file system, applied to a backup device, characterized in that, The method includes: In response to a preset backup trigger condition, a set of data change logs within a target time window is determined. The set of data change logs contains multiple data change log records, and each data change log record contains the logical address information of the data block in which the data change occurred. By traversing the data change log set, the total number of times a logical address appears as a change object in the data change log set is determined, and the total number of times is determined as the update frequency value of the data block corresponding to the logical address; Based on a preset frequency threshold, the data blocks are divided into a low-frequency update set and a high-frequency update set; Based on the logical address information of each data block in the low-frequency update set, determine the first incremental data corresponding to the low-frequency update set; After all the first incremental data has been acquired, the second incremental data corresponding to the high-frequency update set is determined based on the logical address information of each data block in the high-frequency update set. Write the first incremental data and the second incremental data into the incremental backup file.

2. The method of claim 1, wherein, The step of dividing the data block into a low-frequency update set and a high-frequency update set according to a preset frequency threshold specifically includes: Based on the file system metadata corresponding to the data block, obtain the number of hard links associated with other directory entries for each data block's inode; If the arithmetic mean of the number of hard links is greater than a preset dependency threshold, the probability that each data block is concurrently written to disk with other data blocks in the same transaction log within a preset historical period is calculated as the concurrency aggregation degree. After normalizing the number of hard links and the degree of concurrent aggregation, a weighted calculation is performed using preset weight parameters to obtain a concurrent association weight coefficient that characterizes the risk of each data block being associated and modified. By sorting the sequence of concurrent association weight coefficients corresponding to all data blocks according to their numerical values, the boundary point value with the largest difference between adjacent weight coefficients is identified, and the boundary point value is used as the updated preset frequency threshold. According to the preset frequency threshold, data blocks whose update frequency value is less than the preset frequency threshold are classified into a low-frequency update set, and data blocks whose update frequency value is greater than or equal to the preset frequency threshold are classified into a high-frequency update set.

3. The method of claim 2, wherein, After the step of obtaining the number of hard links associated with other directory entries for each inode to which the data block belongs based on the file system metadata corresponding to the data block, the method further includes: If the arithmetic mean of the number of hard links is less than or equal to the preset dependency threshold, retrieve the physical block mapping table of the logical address corresponding to each data block on the physical storage medium. Calculate the spatial offset of adjacent data blocks at each logical address on the physical storage medium, and define the distribution ratio of data blocks with spatial offsets greater than a preset continuity threshold as spatial sparsity; The update frequency value and the spatial discrete sparsity are weighted to obtain the addressing index corresponding to each data block; The threshold value at which the difference between adjacent indices in the sequence of addressing indices reaches its maximum value is established as an auxiliary threshold. Data blocks with addressing indices less than the auxiliary threshold are combined into the low-frequency update set, and data blocks with addressing indices greater than or equal to the auxiliary threshold are combined into the high-frequency update set.

4. The method of claim 1, wherein, Before the step of determining the second incremental data corresponding to the high-frequency update set based on the logical address information of each data block in the high-frequency update set after all the first incremental data has been acquired, the method further includes: Within the time period of acquiring the first incremental data corresponding to the low-frequency update set, based on the preferred high-frequency target server corresponding to each data block in the high-frequency update set, the current network response latency and disk I / O wait queue length of each preferred high-frequency target server are extracted. Calculate the node response latency multiplier for each of the preferred high-frequency target servers based on the current network response latency and the disk I / O wait queue length; Based on the target logical address corresponding to the preferred high-frequency target server that generates the node response latency multiplier, alternative node addressing is performed to determine the alternative server to replace the preferred high-frequency target server.

5. The method of claim 4, wherein, The step of determining a replacement server for the preferred high-frequency target server by addressing the target logical address corresponding to the preferred high-frequency target server that generates the node response latency multiplier specifically includes: Using the metadata dictionary and the target logical address corresponding to the preferred high-frequency target server that generates the node response latency multiplier, extract all secondary candidate slave servers that store the data block replicas corresponding to the target logical address, and obtain the latest replica synchronization timestamp of each secondary candidate slave server. After obtaining the last health timestamp of the preferred high-frequency target server, the secondary candidate slave servers whose replica synchronization timestamp is later than or equal to the last health timestamp are combined into a safe alternative cluster. The secondary candidate server with the highest historical average transmission rate is selected from the security candidate cluster as the candidate server. Update the read request node corresponding to the target logical address to the alternative server.

6. The method according to claim 5, characterized in that, After the step of writing the first incremental data and the second incremental data to the incremental backup file, the method further includes: Obtain the delivery timestamp of the service response message sent by the alternative server to the backup device when transmitting the second incremental data; Based on the delivery timestamp, an out-of-band alignment query request is initiated to the master control component of the distributed cluster corresponding to the target logical address; The system receives feedback from the master control component that the master node within the interval between the replica synchronization timestamp and the delivery timestamp has not issued a change log. If the master node does not issue a change log that is not empty, it is determined that there are unsynchronized data segments in the second incremental data, and a data correction strategy for the unsynchronized data segments is triggered.

7. The method according to claim 6, characterized in that, The step of triggering the data correction strategy for the unsynchronized data segment specifically includes: Extract the physical offset range corresponding to the second incremental data in the incremental backup file, and use it as the data area to be repaired; Based on the master node's failure to issue change logs, extract the missing data segments that were not synchronized to the alternative server within the specified interval and the corresponding block offset address of the missing data segments. The corresponding old data stream is located within the data area to be repaired by addressing the offset address within the block. The missing data fragments are used to overwrite the data content of the old data stream, and a block checksum is calculated based on the overwritten data area to be repaired. The block checksum is used to update the integrity verification information of the incremental backup file.

8. A backup device, characterized in that, The backup device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the backup device to perform the method as described in any one of claims 1-7.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the backup device, the backup device performs the method as described in any one of claims 1-7.

Images