Hadoop cluster incremental backup and recovery method based on data block optimization

Abstract

The application provides a Hadoop cluster incremental backup recovery method based on data block optimization, relates to the technical field of data management, and comprises the following steps: acquiring distribution state information and historical change records of data blocks in a distributed file system, extracting space-time correlation features, and constructing a data block dependency graph containing dependency direction and dependency strength; identifying data blocks that have changed, performing topological sorting, and identifying key path data blocks and data block clusters that are not dependent on each other; calculating distribution entropy values according to the distribution positions of the replicas, and determining backup granularity types; generating a backup execution sequence according to the topological sorting results, preferentially backing up the key path data blocks, and parallelly backing up the data block clusters that are not dependent on each other, and generating backup metadata containing dependency relationship identifiers and distribution entropy values; and performing dependency consistency checking after recovery based on the dependency relationship identifiers. The application improves backup and recovery efficiency, guarantees data consistency, and reduces storage costs.

Description

Technical Field

[0001] This invention relates to data management technology, and more particularly to an incremental backup and recovery method for Hadoop clusters based on data block optimization. Background Technology

[0002] The Hadoop Distributed File System has become a core infrastructure for enterprises to store and process massive amounts of data. In the operation and maintenance of Hadoop clusters, data backup and recovery are key technologies for ensuring data security and business continuity. However, existing incremental backup technologies for Hadoop clusters still have many shortcomings.

[0003] Existing incremental backup methods typically back up data blocks as independent units, ignoring the logical dependencies and spatiotemporal relationships between them. This simplistic approach makes it impossible to accurately restore the dependencies between data blocks during data recovery, potentially leading to data consistency issues, especially when dealing with highly correlated business data, where the integrity of the recovered data is difficult to guarantee.

[0004] Existing technologies lack optimization strategies for backup order when performing incremental backups, often employing sequential scanning or random selection to process changed data blocks. This approach fails to fully utilize the dependencies between data blocks and parallel processing capabilities, resulting in unnecessary waiting time and resource idleness during the backup process. It fails to fully leverage the parallel processing advantages of distributed systems, leading to overall low backup efficiency.

[0005] Existing incremental backup solutions typically employ a fixed backup granularity, failing to dynamically adjust backup strategies based on the actual distribution characteristics and change patterns of data blocks. In Hadoop clusters, the replica distribution of different data blocks varies significantly. A fixed-granularity backup approach may result in some data blocks being over-backed up while others are under-backed up, wasting storage resources, increasing network transmission burden, and lacking an adaptive mechanism for optimizing strategies based on historical backup experience. Summary of the Invention

[0006] This invention provides an incremental backup and recovery method for Hadoop clusters based on data block optimization, which can solve the problems in the prior art.

[0007] A first aspect of this invention provides a Hadoop cluster incremental backup and recovery method based on data block optimization, comprising:

[0008] Obtain the distribution status information and historical change records of data blocks in the distributed file system, extract the spatiotemporal correlation features between data blocks, and construct a data block dependency graph that includes dependency direction and dependency strength;

[0009] Identify the changed data blocks, perform topological sorting of the changed data blocks according to the data block dependency graph, and identify critical path data blocks and clusters of independent data blocks;

[0010] The distribution entropy value is calculated based on the replica distribution location of the changed data blocks, and the backup granularity type is determined based on the distribution entropy value;

[0011] A backup execution sequence is generated based on the topological sorting result. The critical path data blocks are backed up first, and the independent data block clusters are backed up in parallel. At the same time, backup is performed according to the backup granularity type, and backup metadata containing dependency identifiers and distribution entropy values ​​is generated.

[0012] Dependency consistency is verified after recovery based on the dependency identifiers in the backup metadata, and the dependency strength in the data block dependency graph is adaptively adjusted using the distribution entropy value change trend of multiple backup cycles.

[0013] In one optional implementation, the steps of obtaining the distribution status information and historical change records of data blocks in the distributed file system, extracting the spatiotemporal correlation features between data blocks, and constructing a data block dependency graph containing dependency direction and dependency strength include:

[0014] Based on the distribution status information, the physical distribution location relationship and logical read / write access relationship of the data blocks are extracted as spatial association features; based on the historical change records, the change time sequence and change propagation path of the data blocks are extracted as temporal association features.

[0015] By coupling the spatial correlation features with the temporal correlation features, we can identify pairs of correlated data blocks that exhibit temporal correlation changes and spatial dependencies.

[0016] The dependency direction is determined based on the order of change time and read / write access relationship of the first and second data blocks in the associated data block pair. The dependency strength is determined based on the frequency and stability of the associated data block pair in multiple backup cycles. A data block dependency graph is constructed, where nodes represent data blocks, directed edges represent the dependency direction, and edge weights represent the dependency strength.

[0017] In one optional implementation, the step of coupling the spatial correlation features with the temporal correlation features to identify correlated data block pairs that exhibit temporal correlation changes and spatial dependencies includes:

[0018] For each data block, a spatial proximity matrix is ​​constructed based on the physical distribution location relationship, and a read-write dependency matrix is ​​constructed based on the logical read-write access relationship; the data block pairs that actually underwent association changes in historical backup cycles are statistically analyzed, and the contribution ratio of spatial factors and read-write factors to association changes is calculated. Based on the contribution ratio, the fusion weight of the spatial proximity matrix and the read-write dependency matrix is ​​dynamically determined to generate a spatial association strength matrix;

[0019] Based on the change time series and change propagation path, a change causal chain is extracted. An adaptive time window threshold is calculated by analyzing the historical change propagation speed of data blocks within the change causal chain. Data block pairs with change time intervals less than the adaptive time window threshold and located on the same change causal chain are marked as time-related data block pairs.

[0020] The data block pairs with a correlation strength exceeding a preset spatial threshold in the spatial correlation strength matrix are intersected with the time-related data block pairs. The intersection results are then subjected to multi-period stability verification. The frequency of occurrence of the data block pairs in multiple consecutive backup periods is counted. When the frequency of occurrence exceeds a preset stability threshold, the data block pairs are confirmed as related data block pairs.

[0021] In one optional implementation, the steps of identifying changed data blocks, performing topological sorting of the changed data blocks according to the data block dependency graph, and identifying critical path data blocks and clusters of independent data blocks include:

[0022] The distribution state information is compared with the pre-stored baseline distribution state information. When there is a difference between the current state characteristics of a data block and the baseline state characteristics, the corresponding data block is marked as a changed data block.

[0023] Based on the directed edges in the data block dependency graph, calculate the in-degree and out-degree values ​​of each changed data block; mark changed data blocks with in-degree values ​​greater than a preset in-degree threshold or out-degree values ​​greater than a preset out-degree threshold as critical path data blocks;

[0024] The modified data blocks are topologically sorted in the data block dependency graph to obtain a sequence of data blocks that satisfy the dependency relationship; a set of data blocks that are not connected by directed edges to each other is identified in the data block sequence, and the data block set is determined as a data block cluster that is independent of each other; the data blocks in the data block cluster are sorted in descending order according to their global centrality index in the data block dependency graph, and backed up in sequence.

[0025] In one optional implementation, the step of calculating a distribution entropy value based on the replica distribution location of the changed data block, and determining the backup granularity type based on the distribution entropy value, includes:

[0026] Obtain the distribution locations of all replicas of each modified data block in the distributed file system, and calculate the distribution entropy value of the modified data block based on the number of replicas distributed across each storage node;

[0027] The in-degree value of the changed data block is extracted from the data block dependency graph, and the distribution threshold is dynamically adjusted based on the in-degree value, wherein when the in-degree value indicates that the data block is the critical path data block, the distribution threshold is reduced;

[0028] When the distribution entropy value is higher than the adjusted distribution threshold, the backup granularity type is determined to be block-level backup; when the distribution entropy value is lower than the adjusted distribution threshold, the backup granularity type is determined to be file-level backup.

[0029] For the independent data block clusters, the variance of the distribution entropy value of each data block within the data block cluster is calculated; when the variance of the distribution entropy value exceeds a preset difference threshold, abnormal data blocks whose distribution entropy value deviates from the mean are identified, and the backup granularity type of the abnormal data blocks is adjusted to data block-level backup, while the backup granularity type of the remaining data blocks is uniformly determined according to the average distribution entropy value within the cluster.

[0030] In one optional implementation, the steps of generating a backup execution sequence according to the topology sorting result, prioritizing the backup of the critical path data blocks and backing up the independent data block clusters in parallel, and simultaneously performing backups according to the backup granularity type, and generating backup metadata containing dependency identifiers and distribution entropy values ​​include:

[0031] Based on the topology sorting results, a hierarchical backup queue is constructed, placing the critical path data blocks in the priority level and the independent data block clusters in the parallel level; the backup resource ratio is dynamically calculated based on the number of critical path data blocks and the sum of their dependencies, and corresponding system I / O bandwidth and storage resources are allocated to the priority level to generate a backup execution sequence;

[0032] Extract the backup granularity type of the current data block to be backed up from the backup execution sequence. When the backup granularity type is block-level backup, perform backup operations on all copies. When the backup granularity type is file-level backup, perform backup operations only on the primary copy.

[0033] During the backup operation, dependency direction and dependency strength are extracted from the data block dependency graph. The dependency direction is converted into a dependency relationship identifier, and a backup metadata record containing the dependency relationship identifier, dependency strength, and distribution entropy value is created.

[0034] Once the priority-level backup is complete, initiate the parallel-level backup; monitor the backup progress, and if the backup time for a data block cluster exceeds the predicted time threshold, move its incomplete data blocks to the head of the queue and increase the number of threads.

[0035] A metadata index tree is constructed based on the dependency relationship identifier, and repeated dependency patterns are compressed and encoded to generate backup metadata files.

[0036] In one optional implementation, the steps of performing dependency consistency verification after recovery based on the dependency identifiers in the backup metadata, and adaptively adjusting the dependency strength in the data block dependency graph using the distribution entropy value change trend over multiple backup cycles, include:

[0037] After data recovery is complete, the dependency identifiers of all data blocks are extracted from the backup metadata to construct a dependency verification graph of the recovered data blocks;

[0038] Traverse each dependency edge in the data block dependency verification graph, and detect the actual read / write access relationship between the predecessor and successor data blocks connected by the dependency edge in the recovered file system. When the actual read / write access relationship is inconsistent with the dependency direction indicated by the dependency relationship identifier, it is marked as a dependency consistency anomaly.

[0039] For data blocks marked as having dependency consistency anomalies, the distribution entropy value sequence of the data block in the current backup cycle and historical backup cycles is extracted from the backup metadata, and the change trend coefficient of the distribution entropy value sequence is calculated; based on the inverse correlation between the change trend coefficient and the direction of change of the distribution entropy value, the dependency strength of the corresponding dependency edge in the dependency graph of the data block is adjusted;

[0040] The adjusted dependency strength is updated to the data block dependency graph for topology sorting and critical path data block identification in the next backup cycle.

[0041] This invention constructs a data block dependency graph that includes dependency direction and dependency strength, which can accurately identify the relationship between data blocks. This provides a precise dependency analysis basis for subsequent incremental backups, avoiding the problem of incomplete or redundant backups caused by ignoring data block dependencies in traditional methods, and improving the accuracy and reliability of backups.

[0042] This invention identifies critical path data blocks and independent data block clusters by performing topological sorting on changed data blocks, and determines the backup granularity by combining the distribution entropy value. This achieves a strategy of prioritizing the backup of critical data blocks and backing up independent data block clusters in parallel, which significantly improves backup efficiency and shortens the backup time window. At the same time, it dynamically adjusts the backup granularity according to the data distribution characteristics, reducing network transmission overhead and storage resource consumption.

[0043] This invention ensures the integrity and consistency of data recovery by recording dependency identifiers in backup metadata and performing dependency consistency verification after recovery. At the same time, it adaptively adjusts the dependency strength by utilizing the distribution entropy value change trend of multiple backup cycles, so that the data block dependency graph can dynamically adapt to the actual changes in cluster data, thereby improving the system's adaptability and long-term operational stability. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the incremental backup and recovery method for Hadoop clusters based on data block optimization, as described in an embodiment of the present invention.

[0045] Figure 2 Flowchart for constructing a data block dependency graph. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0048] Figure 1 This is a flowchart illustrating the incremental backup and recovery method for Hadoop clusters based on data block optimization, as described in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0049] Obtain the distribution status information and historical change records of data blocks in the distributed file system, extract the spatiotemporal correlation features between data blocks, and construct a data block dependency graph that includes dependency direction and dependency strength;

[0050] Identify the changed data blocks, perform topological sorting of the changed data blocks according to the data block dependency graph, and identify critical path data blocks and clusters of independent data blocks;

[0051] The distribution entropy value is calculated based on the replica distribution location of the changed data blocks, and the backup granularity type is determined based on the distribution entropy value;

[0052] A backup execution sequence is generated based on the topological sorting result. The critical path data blocks are backed up first, and the independent data block clusters are backed up in parallel. At the same time, backup is performed according to the backup granularity type, and backup metadata containing dependency identifiers and distribution entropy values ​​is generated.

[0053] Dependency consistency is verified after recovery based on the dependency identifiers in the backup metadata, and the dependency strength in the data block dependency graph is adaptively adjusted using the distribution entropy value change trend of multiple backup cycles.

[0054] In one optional implementation, the steps of obtaining the distribution status information and historical change records of data blocks in the distributed file system, extracting the spatiotemporal correlation features between data blocks, and constructing a data block dependency graph containing dependency direction and dependency strength include:

[0055] Based on the distribution status information, the physical distribution location relationship and logical read / write access relationship of the data blocks are extracted as spatial association features; based on the historical change records, the change time sequence and change propagation path of the data blocks are extracted as temporal association features.

[0056] By coupling the spatial correlation features with the temporal correlation features, we can identify pairs of correlated data blocks that exhibit temporal correlation changes and spatial dependencies.

[0057] The dependency direction is determined based on the order of change time and read / write access relationship of the first and second data blocks in the associated data block pair. The dependency strength is determined based on the frequency and stability of the associated data block pair in multiple backup cycles. A data block dependency graph is constructed, where nodes represent data blocks, directed edges represent the dependency direction, and edge weights represent the dependency strength.

[0058] Combination Figure 2The flowchart for constructing the data block dependency graph is used to illustrate this process. In the distributed file system (DFS), the distribution status information of all data blocks is obtained through the DFS metadata management service. This distribution status information includes the physical storage node identifier, storage path, replica distribution location, and logical access records for each data block. For data block BLOCK_A, its physical distribution location is recorded as disk partition DISK_03 on storage node NODE_001, with two replicas existing on storage nodes NODE_005 and NODE_012. Extracting the logical read / write access relationship of this data block from the access log reveals that during the time period from timestamp 1609459200 to 1609459800, application process APP_SERVICE_01 performed a read operation on BLOCK_A, and subsequently performed a write operation on data block BLOCK_B at timestamp 1609459850. This read-then-write access pattern constitutes the spatial association characteristic between data blocks.

[0059] To extract the physical distribution relationship, a physical distance metric is calculated between data blocks. When BLOCK_A and BLOCK_C are located on the same disk partition of the same storage node, their physical distance metric is set to 1, representing the strongest physical association. When two data blocks are located on different disk partitions of the same storage node, the physical distance metric is set to 2. If data blocks are distributed across different storage nodes but belong to the same rack, the physical distance metric is set to 3. For data blocks distributed across racks, the physical distance metric is set to 4. This metric quantifies the physical distribution relationship of the data blocks.

[0060] Historical change records are extracted from the distributed file system's change log service. This log records the modification timestamp, modification type, trigger source, and scope of impact for each data block. For the BLOCK_D data block, its historical change record shows a content update operation occurred at timestamp 1609460100, with the change type being data write and the trigger source being the application process APP_SERVICE_02. Tracing the propagation path of this change reveals that at timestamp 1609460150, the data block numbered BLOCK_E also underwent a change, and this change operation was triggered by the same application process. By analyzing consecutive change log entries, a change propagation chain was identified: the change in BLOCK_D led to a corresponding change in BLOCK_E, which in turn affected the data state of BLOCK_F. This change propagation path constitutes the core element of the time-related feature.

[0061] The extraction of change time series is achieved by arranging the change records of a single data block within a continuous time window in chronological order. For the BLOCK_G data block, the change time series within ten backup cycles is extracted, namely timestamps 1609460200, 1609467400, 1609474600, 1609481800, 1609489000, 1609496200, 1609503400, 1609510600, 1609517800, and 1609525000. By calculating the interval between adjacent change time points, it is found that this data block changes at a fixed period of 7200 seconds, and this periodicity is used as a quantitative indicator of time correlation characteristics.

[0062] The spatiotemporal correlation coupling analysis is achieved by constructing an correlation matrix. All data block pairs are traversed. For any two data blocks BLOCK_M and BLOCK_N, it is checked whether they simultaneously satisfy both spatial dependency and temporal correlation conditions. The criteria for spatial dependency include that the physical distance metric between the two data blocks is less than or equal to 3, or that there are consecutive read / write access records from the same application process. The criteria for temporal correlation are that the time difference between the changes of the two data blocks is within a preset time window threshold, which is set to 300 seconds based on the actual operating characteristics of the system. When BLOCK_M changes at timestamp 1609530000 and BLOCK_N changes at timestamp 1609530180, the time difference is 180 seconds, satisfying the temporal correlation condition. Furthermore, if the application process APP_SERVICE_03 accesses BLOCK_N immediately after changing BLOCK_M, then this data block pair is identified as a correlated data block pair.

[0063] The dependency direction is determined based on the order of changes and read / write access relationships within the associated data block pair. In the associated data block pair BLOCK_P and BLOCK_Q, ​​the change timestamp for BLOCK_P is 1609535000, and for BLOCK_Q it is 1609535200. Access logs show that the application process reads the content of BLOCK_P at timestamp 1609535050 and then uses that read data to update BLOCK_Q at timestamp 1609535180. Therefore, the dependency direction is determined to be from BLOCK_P to BLOCK_Q, ​​indicating that BLOCK_Q depends on the data content of BLOCK_P. If the access logs show that the application process only performs independent write operations on the two data blocks without any read association, the dependency relationship is determined by analyzing the similarity of the data block content. For data block pairs with a content similarity higher than the threshold of 0.8, the data block with the earlier change time is designated as the dependency source.

[0064] Dependency strength is calculated based on the frequency and stability of associated data block pairs across multiple backup periods. The observation period is set to twenty consecutive backup periods, and the number of times each associated data block pair experiences simultaneous associated changes within these periods is counted. For the associated data block pair BLOCK_R and BLOCK_S, simultaneous associated changes occurred eighteen times across the twenty backup periods, with a frequency of eighteen. Stability is measured by calculating the standard deviation of the associated change time intervals. For this data block pair, the time intervals during the eighteen associated changes were 150 seconds, 165 seconds, 148 seconds, 170 seconds, and 155 seconds, respectively. The average of these intervals is 158 seconds, and the standard deviation is 8 seconds. A smaller standard deviation indicates a more stable association. The dependency strength value is calculated by dividing the frequency by the total number of observation periods to obtain the base weight, then adjusting it based on the standard deviation. When the standard deviation is less than 10% of the average time interval, the base weight is multiplied by an enhancement factor of 1.5 to obtain the final dependency strength value.

[0065] The data block dependency graph is stored using a directed weighted graph data structure. Each node in the graph corresponds to a data block, and node attributes include metadata information such as data block identifier, physical location, capacity, and access frequency. Directed edges connect data block nodes with dependencies, the direction of the edge indicates the dependency direction, and the weight of the edge is the dependency strength value calculated earlier. In a specific graph example, node BLOCK_T points to node BLOCK_U via a directed edge with a weight of 0.9, indicating that BLOCK_U is strongly dependent on BLOCK_T. A single data block node has multiple incoming and outgoing edges. Node BLOCK_V has three incoming edges from BLOCK_W, BLOCK_X, and BLOCK_Y, with weights of 0.7, 0.85, and 0.6 respectively, and two outgoing edges pointing to BLOCK_Z and BLOCK_AA, with weights of 0.75 and 0.65 respectively. After the graph is constructed, it is stored in a graph database, supporting fast query and path analysis operations based on dependencies.

[0066] This invention constructs a comprehensive graph that reflects the dependencies between data blocks by simultaneously considering the physical distribution and logical read / write access relationships of data blocks, combined with the change sequence and propagation path over time. This enables the backup and recovery process to accurately grasp the correlation between data blocks, thereby improving the accuracy and efficiency of backup and recovery.

[0067] In one optional implementation, the step of coupling the spatial correlation features with the temporal correlation features to identify correlated data block pairs that exhibit temporal correlation changes and spatial dependencies includes:

[0068] For each data block, a spatial proximity matrix is ​​constructed based on the physical distribution location relationship, and a read-write dependency matrix is ​​constructed based on the logical read-write access relationship; the data block pairs that actually underwent association changes in historical backup cycles are statistically analyzed, and the contribution ratio of spatial factors and read-write factors to association changes is calculated. Based on the contribution ratio, the fusion weight of the spatial proximity matrix and the read-write dependency matrix is ​​dynamically determined to generate a spatial association strength matrix;

[0069] Based on the change time series and change propagation path, a change causal chain is extracted. An adaptive time window threshold is calculated by analyzing the historical change propagation speed of data blocks within the change causal chain. Data block pairs with change time intervals less than the adaptive time window threshold and located on the same change causal chain are marked as time-related data block pairs.

[0070] The data block pairs with a correlation strength exceeding a preset spatial threshold in the spatial correlation strength matrix are intersected with the time-related data block pairs. The intersection results are then subjected to multi-period stability verification. The frequency of occurrence of the data block pairs in multiple consecutive backup periods is counted. When the frequency of occurrence exceeds a preset stability threshold, the data block pairs are confirmed as related data block pairs.

[0071] For example, by accessing the metadata management service of the distributed file system, the storage node identifier, disk partition path, and replica distribution information of each data block are extracted. For any two data blocks, their physical distance metric is calculated. Specifically, data blocks on the same storage node and the same disk partition have a physical distance metric of 1; data blocks on different disk partitions on the same storage node have a physical distance metric of 2; data blocks on different storage nodes within the same rack have a physical distance metric of 3; and data blocks distributed across racks have a physical distance metric of 4. The physical distance metric is converted into a proximity weight using a reciprocal relationship, i.e., the proximity weight equals (5 - physical distance metric) ÷ 4, so that the proximity weight of data block pairs on the same partition is 1, and the proximity weight of data block pairs across racks is 0.25. All data block pairs are traversed, and the proximity weight values ​​are filled into a spatial proximity matrix. This matrix is ​​a symmetric matrix, where the row and column indices correspond to the data block identifiers, and the matrix element values ​​are the proximity weights. For example, data block BLOCK_A is located in disk partition DISK_03 of storage node NODE_001, and data block BLOCK_B is located in disk partition DISK_05 of storage node NODE_001. The physical distance metric between the two is 2, and the corresponding proximity weight is 0.75. This value is stored in the matrix at the position of BLOCK_A row BLOCK_B column and BLOCK_B row BLOCK_A column.

[0072] The read-write dependency matrix is ​​constructed based on logical read-write access relationships. All application processes' read-write operation records for data blocks are extracted from the distributed file system's access log service. Each record includes a timestamp, application process identifier, data block identifier, operation type, and operation duration. For any two data blocks, it is checked whether the same application process exhibits an access pattern where it reads the first data block and then writes to the second data block within a short time window. This time window threshold is set to 600 seconds. If the same application process performs a read operation on data block X at timestamp T, and then performs a write operation on data block Y before timestamp T+600 seconds later, a read-write dependency relationship from X to Y is considered to exist. The frequency of this read-write dependency relationship is counted across all backup periods, and the read-write dependency weight is obtained by dividing the frequency by the total number of observation periods. The read-write dependency matrix is ​​an asymmetric matrix; the row index represents the read data block, the column index represents the write data block, and the matrix element values ​​are the read-write dependency weights. Application process APP_SERVICE_01 reads BLOCK_C at timestamp 1609459200 and writes BLOCK_D at timestamp 1609459450, with a time interval of 250 seconds, satisfying the time window condition. In observations over 20 consecutive backup cycles, this read-write pattern occurred 16 times, with a read-write dependency weight of 16 ÷ 20 = 0.8. This value is stored in the BLOCK_C row and BLOCK_D column of the read-write dependency matrix.

[0073] The contribution ratio is calculated by statistically analyzing the weight distribution characteristics of data block pairs that have undergone related changes in historical backup cycles within the spatial proximity matrix and read / write dependency matrix. Data block pairs that have changed simultaneously within 10 consecutive backup cycles with a time interval of less than 300 seconds are extracted from the change log. This time interval threshold is determined based on the historical average speed of system change propagation. For each data block pair with actual related changes, its weight value in the spatial proximity matrix and read / write dependency matrix is ​​queried. The sum of the spatial proximity weights and the sum of the read / write dependency weights for all data block pairs with actual related changes are calculated. Spatial factor contribution = sum of spatial proximity weights ÷ sum of the two weight sums; read / write factor contribution = sum of read / write dependency weights ÷ sum of the two weight sums, where the sum of the two weight sums is the sum of the spatial proximity weights plus the sum of the read / write dependency weights. For example, in the observed 100 pairs of data block pairs with actual related changes, the sum of the spatial proximity weights is 65, the sum of the read / write dependency weights is 55, and the total is 120. The contribution of spatial factors is approximately 0.54 (65 ÷ 120), and the contribution of read / write factors is approximately 0.46 (55 ÷ 120). This contribution ratio is used as the fusion weight. Each element in the spatial proximity matrix is ​​multiplied by 0.54, and the corresponding element in the read / write dependency matrix is ​​multiplied by 0.46. The two weighted results are then added to obtain the element values ​​of the spatial association strength matrix. The spatial association strength matrix inherits the asymmetric characteristic of the read / write dependency matrix, with row and column indices corresponding to data block identifiers.

[0074] For each data block, all change timestamps within the continuous backup cycle are extracted from the change log and arranged chronologically to form a change time series. By tracking the trigger source identifier and associated fields in the change log, cascading change operations triggered by the same application process or transaction are identified. When data block X changes at timestamp T1, and data block Y changes at timestamp T2, and both are caused by the same trigger source, and the time difference between T2 and T1 is less than a preset propagation delay limit, the change path from X to Y is recorded as a causal edge. Causal edges are recursively constructed until no further expansion is possible, forming a complete change causal chain. This causal chain is stored in a directed acyclic graph structure, where nodes are data blocks, edges represent causal relationships, and edge weights are the change time intervals. For example, data block BLOCK_E changes at timestamp 1609460100, with the trigger source being application process APP_SERVICE_02; data block BLOCK_F changes at timestamp 1609460150, also with the trigger source being APP_SERVICE_02, and the time interval is 50 seconds. Further investigation revealed that BLOCK_G changed at timestamp 1609460210, with the trigger source still being APP_SERVICE_02, and the time interval between BLOCK_G and BLOCK_F being 60 seconds. The constructed causal chain is from BLOCK_E to BLOCK_F and then to BLOCK_G, with edge weights of 50 seconds and 60 seconds respectively.

[0075] For each changed causal chain, the time interval values ​​of all causal edges in the chain are extracted, and the average and standard deviation of these time intervals are calculated. The time window threshold is set to the average + 2 × standard deviation, which covers approximately 95% of normal change propagation delays. When the average propagation delay of a causal chain is 180 seconds and the standard deviation is 30 seconds, the corresponding time window threshold is 180 + 2 × 30 = 240 seconds. For different changed causal chains, the time window threshold is dynamically adjusted according to their historical propagation speed; fast-propagating causal chains use a smaller threshold, and slow-propagating causal chains use a larger threshold. In practical applications, to avoid missing associations due to an excessively small threshold, the lower limit of the time window threshold is set to 120 seconds, and the upper limit is set to 600 seconds. Calculation results exceeding this range are truncated.

[0076] The labeling of time-related data block pairs is achieved by examining the change time interval and causal chain affiliation of the data block pairs. All data block pairs that changed within the same backup cycle are traversed, and the absolute value of the difference between the change timestamps of the two data blocks is calculated. The pair is then queried to determine if it lies on the same change causal chain. This is done by performing a path search in the directed acyclic graph of the change causal chain to determine if a path exists from the first data block to the second data block or vice versa. If the change time interval is less than the adaptive time window threshold corresponding to the causal chain and the path exists, the data block pair is marked as a time-related data block pair. For example, data blocks BLOCK_H and BLOCK_I changed at timestamps 1609465000 and 1609465200, a time interval of 200 seconds. The query finds that they lie on the same change causal chain, whose adaptive time window threshold is 240 seconds. Since 200 seconds is less than 240 seconds and the path exists, BLOCK_H and BLOCK_I are marked as a time-related data block pair. The tagging information is stored in a set of time-associated data block pairs, where each element is a tuple of identifiers for the data block pairs.

[0077] The spatial association strength matrix is ​​traversed to extract data block pairs with association strength exceeding a preset spatial threshold of 0.5, indicating that only spatial associations of medium to high strength are retained. The extracted data block pair identifiers are matched against elements in the set of temporally associated data block pairs. If a data block pair exists in both sets, it is added to the intersection result set. This intersection result represents candidate associated data block pairs that simultaneously satisfy both strong spatial and temporal association conditions. For example, the association strength between BLOCK_J and BLOCK_K in the spatial association strength matrix is ​​0.72, exceeding the threshold of 0.5. Simultaneously, records for BLOCK_J and BLOCK_K also exist in the set of temporally associated data block pairs. This data block pair is added to the intersection result set and proceeds to the subsequent stability verification stage.

[0078] Multi-cycle stability verification is achieved by counting the frequency of occurrence of data block pairs across multiple consecutive backup cycles. For each data block pair in the intersection result set, data from 15 consecutive backup cycles is backtracked from the historical change log. It is checked whether the data block pair simultaneously satisfies both spatial and temporal association conditions within each backup cycle; if so, the count is incremented. The stability threshold is set to 12, requiring the data block pair to appear at least 12 times in the 15 cycles, representing an 80% frequency. When the frequency of a data block pair exceeds the stability threshold, it is confirmed as a stable associated data block pair and written into the final associated data block pair result set. If the frequency does not reach the threshold, the association relationship of the data block pair is considered unstable, and it is excluded from the result set. For example, data block pairs BLOCK_L and BLOCK_M satisfy both spatial and temporal association conditions in 13 of the 15 backtracked backup cycles, with a frequency of 13, exceeding the threshold of 12. The data block pair was identified as a related data block pair, and its dependency and strength information were subsequently used to construct the directed edges and weights of the data block dependency graph. The entire stability verification process adopted a sliding window strategy. After each new backup cycle was completed, the window slid forward one cycle, removing the data from the earliest cycle and incorporating the data from the latest cycle, continuously updating the stability evaluation results of the related data block pair.

[0079] This invention employs dynamic fusion of spatial proximity matrix and read / write dependency matrix, combined with adaptive time window threshold and multi-cycle stability verification, to accurately identify the real data block association relationship, avoid misjudgment and omission, and improve the accuracy and reliability of data block dependency relationship identification.

[0080] In one optional implementation, the steps of identifying changed data blocks, performing topological sorting of the changed data blocks according to the data block dependency graph, and identifying critical path data blocks and clusters of independent data blocks include:

[0081] The distribution state information is compared with the pre-stored baseline distribution state information. When there is a difference between the current state characteristics of a data block and the baseline state characteristics, the corresponding data block is marked as a changed data block.

[0082] Based on the directed edges in the data block dependency graph, calculate the in-degree and out-degree values ​​of each changed data block; mark changed data blocks with in-degree values ​​greater than a preset in-degree threshold or out-degree values ​​greater than a preset out-degree threshold as critical path data blocks;

[0083] The modified data blocks are topologically sorted in the data block dependency graph to obtain a sequence of data blocks that satisfy the dependency relationship; a set of data blocks that are not connected by directed edges to each other is identified in the data block sequence, and the data block set is determined as a data block cluster that is independent of each other; the data blocks in the data block cluster are sorted in descending order according to their global centrality index in the data block dependency graph, and backed up in sequence.

[0084] For example, the metadata management service of the distributed file system collects the current state characteristics of all data blocks at fixed time intervals. These state characteristics include the data block's hash checksum, last modified timestamp, file size, and a list of replica distribution nodes. The baseline distribution state information is stored in the persistent storage layer, recording a snapshot of the state characteristics of each data block at the time the previous backup cycle was completed. The comparison process checks the consistency between the current state characteristics and the baseline state characteristics item by item. The hash checksum is a 64-byte hexadecimal string calculated using the SHA-256 algorithm. A complete match between the current and baseline values ​​is considered a match; any difference in character position indicates a discrepancy. The last modified timestamp is stored in UNIX timestamp format with second-level precision; a difference is considered when the current timestamp is greater than the baseline timestamp. The file size is stored in bytes; a difference is considered when the current value is not equal to the baseline value. The replica distribution node list is an ordered string array; a difference is considered when the number of elements in the current list differs from the baseline list or when the node identifiers at corresponding positions do not match. When any state characteristic of a data block differs, the identifier of that data block is written to the changed data block set. The set is stored using a hash table structure to support fast lookup; the key is the data block identifier, and the value is a difference type marker bitmap. For example, the current hash checksum of data block BLOCK_M is 8f4e3a2b1c9d7e6f5a4b3c2d1e0f9a8b7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f, while the baseline hash checksum is 7a3c5e9b2d4f6a8c0e1b3d5f7a9c2e4b6d8f0a2c4e6b8d0f2a4c6e8b0d2f4a6c, and the two do not match. Simultaneously, the current last modified timestamp is 1732118400, while the baseline timestamp is 1732032000, meaning the current value is greater than the baseline value. The data block is marked as changed, with bits 0 and 1 of the difference type marker bitmap set to 1, representing hash value difference and timestamp difference, respectively.

[0085] The in-degree and out-degree values ​​are calculated based on the directed edge structure in the data block dependency graph. The data block dependency graph is stored in adjacency list format. Each data block, as a graph node, maintains two lists: the out-edge list records the identifiers and edge weights of all target data blocks pointed to by the data block, and the in-edge list records the identifiers and edge weights of all source data blocks pointing to the data block. For each data block in the changed data block set, the number of source data blocks in its in-edge list is counted as the in-degree value, and the number of target data blocks in its out-edge list is counted as the out-degree value. The preset in-degree threshold is set to 3, and the preset out-degree threshold is set to 5. These thresholds are determined based on the statistical distribution of data block dependencies in historical backup cycles. The in-degree threshold covers approximately 85% of ordinary data blocks, and the out-degree threshold covers approximately 80% of ordinary data blocks. When the in-degree value of a changed data block is greater than 3 or the out-degree value is greater than 5, the identifier of that data block is written to the critical path data block set. This set is also stored using a hash table structure, with the data block identifier as the key and the values ​​as tuples of the in-degree and out-degree values. For example, the in-edge list of data block BLOCK_N contains 4 edges from BLOCK_A, BLOCK_B, BLOCK_C, and BLOCK_D, with an in-degree value of 4, which is greater than the preset in-degree threshold of 3. The out-edge list of this data block contains 2 edges pointing to BLOCK_X and BLOCK_Y, with an out-degree value of 2, which does not exceed the preset out-degree threshold of 5. Because the in-degree value exceeds the threshold, BLOCK_N is marked as a critical path data block, and the stored tuple is (4, 2).

[0086] The topological sorting process uses the Kahn algorithm, processing the subgraph formed by changed data blocks in the data block dependency graph. A working queue and a result sequence are initialized. The working queue stores changed data blocks with a current in-degree of zero, and the result sequence records the sorted data blocks. The set of changed data blocks is traversed, calculating the actual in-degree of each changed data block in the subgraph. The actual in-degree only counts the number of incoming edges from other changed data blocks, ignoring incoming edges from unchanged data blocks. Changed data blocks with an actual in-degree of zero are added to the working queue. A data block is retrieved from the working queue, its identifier is appended to the end of the result sequence, and the outgoing edge list of that data block is traversed. For each outgoing edge pointing to another changed data block, the actual in-degree of the target data block is decremented by one. When the actual in-degree of the target data block becomes zero, it is added to the working queue. This process of retrieving and processing is repeated until the working queue is empty. If the length of the result sequence equals the size of the changed data block set, the topological sorting is successful, and the result sequence is the sequence of data blocks that satisfy the dependencies. If the work queue is empty but the length of the result sequence is less than the size of the changed data block set, it indicates that a cycle exists in the subgraph. In this case, the unsorted data blocks are recorded as cycle members, and these data blocks are appended to the end of the result sequence in lexicographical order to complete the sorting. For example, the changed data block set contains 5 data blocks: BLOCK_P, BLOCK_Q, ​​BLOCK_R, BLOCK_S, and BLOCK_T. The outgoing edges of BLOCK_P point to BLOCK_Q and BLOCK_R, the outgoing edges of BLOCK_Q point to BLOCK_S, the outgoing edges of BLOCK_R point to BLOCK_S, the outgoing edges of BLOCK_S point to BLOCK_T, and BLOCK_T has no outgoing edges. Calculating the actual in-degree, BLOCK_P is 0, BLOCK_Q is 1, BLOCK_R is 1, BLOCK_S is 2, and BLOCK_T is 1. The initial work queue contains BLOCK_P. After retrieving BLOCK_P, it is added to the result sequence. The actual in-degrees of BLOCK_Q and BLOCK_R are decremented to 0 and added to the work queue. BLOCK_Q is retrieved and added to the result sequence, and the actual in-degree of BLOCK_S is decremented to 1. BLOCK_R is retrieved and added to the result sequence, and the actual in-degree of BLOCK_S is decremented to 0 and added to the work queue. BLOCK_S is retrieved and added to the result sequence, and the actual in-degree of BLOCK_T is decremented to 0 and added to the work queue. BLOCK_T is retrieved and added to the result sequence. The final result sequence is BLOCK_P, BLOCK_Q, ​​BLOCK_R, BLOCK_S, and BLOCK_T, with a length of 5, consistent with the size of the changed data block set. The topology sorting is successful.

[0087] Construct an undirected connected graph, traverse all data block pairs in the data block sequence, and check if there are directed edges in the data block dependency graph from the first data block to the second data block or from the second data block to the first data block. If a directed edge exists in either direction, add an undirected edge to the undirected connected graph connecting the two data blocks. Use a depth-first search algorithm to identify all connected components in the undirected connected graph. Data blocks within each connected component are directly or indirectly connected to each other through directed edges, and there are no directed edges connecting data blocks in different connected components. Treat each connected component as a cluster of independent data blocks, and store the data block identifiers within the cluster as a set structure. For example, the data block sequence contains 6 data blocks: BLOCK_U, BLOCK_V, BLOCK_W, BLOCK_X, BLOCK_Y, and BLOCK_Z. The dependency graph contains edges: BLOCK_U pointing to BLOCK_V, BLOCK_V pointing to BLOCK_W, and BLOCK_X pointing to BLOCK_Y. There are no directed edges between BLOCK_W and BLOCK_X, or between BLOCK_Z and other data blocks. An undirected connected graph is constructed by adding edges (BLOCK_U, BLOCK_V), (BLOCK_V, BLOCK_W), and (BLOCK_X, BLOCK_Y). Depth-first search identifies three connected components: the first component contains BLOCK_U, BLOCK_V, and BLOCK_W; the second component contains BLOCK_X and BLOCK_Y; and the third component contains BLOCK_Z. These three connected components correspond to three independent data block clusters.

[0088] The global centrality metric is calculated using a variant of the PageRank algorithm, which is executed on the data block dependency graph. Initially, each data block is assigned an equal initial centrality value of 1 divided by the total number of data blocks. During iterative updates, the centrality value of each data block is calculated as follows: Centrality value = Damping coefficient × (Sum of the centrality values ​​÷ Out-degree values ​​of all source data blocks pointing to this data block) + (1 - Damping coefficient) ÷ Total number of data blocks. The damping coefficient is set to 0.85, and the number of iterations is set to 20, or the iteration is terminated early when the sum of the absolute values ​​of the changes in the centrality values ​​of all data blocks in two consecutive iterations is less than 0.0001. For each data block cluster, the global centrality metric values ​​of all data blocks within the cluster are extracted and sorted in descending order of centrality value. If the centrality values ​​are the same, they are arranged in lexicographical order according to the data block identifier. The sorted data block sequence serves as the backup priority sequence for this cluster, with the first data block in the sequence being backed up first and the last data block in the sequence being backed up last. For example, a data block cluster contains three data blocks: BLOCK_V, BLOCK_W, and BLOCK_U. After calculating the global centrality, the centrality values ​​are 0.042 for BLOCK_V, 0.038 for BLOCK_W, and 0.051 for BLOCK_U. The sequences BLOCK_U, BLOCK_V, and BLOCK_W are obtained by sorting the centrality values ​​in descending order. The backup system backs up BLOCK_U, BLOCK_V, and BLOCK_W sequentially according to this sequence, ensuring that data blocks with a significant impact on the overall data flow propagation receive priority backup protection, thus reducing the risk of critical data loss due to backup window time limitations.

[0089] This invention identifies critical path data blocks by calculating the in-degree and out-degree values ​​of data blocks, determines independent data block clusters by using topological sorting, and prioritizes them by combining global centrality indicators, thereby achieving parallel optimization of the backup process and significantly improving backup efficiency.

[0090] In one optional implementation, the step of calculating a distribution entropy value based on the replica distribution location of the changed data block, and determining the backup granularity type based on the distribution entropy value, includes:

[0091] Obtain the distribution locations of all replicas of each modified data block in the distributed file system, and calculate the distribution entropy value of the modified data block based on the number of replicas distributed across each storage node;

[0092] The in-degree value of the changed data block is extracted from the data block dependency graph, and the distribution threshold is dynamically adjusted based on the in-degree value, wherein when the in-degree value indicates that the data block is the critical path data block, the distribution threshold is reduced;

[0093] When the distribution entropy value is higher than the adjusted distribution threshold, the backup granularity type is determined to be block-level backup; when the distribution entropy value is lower than the adjusted distribution threshold, the backup granularity type is determined to be file-level backup.

[0094] For the independent data block clusters, the variance of the distribution entropy value of each data block within the data block cluster is calculated; when the variance of the distribution entropy value exceeds a preset difference threshold, abnormal data blocks whose distribution entropy value deviates from the mean are identified, and the backup granularity type of the abnormal data blocks is adjusted to data block-level backup, while the backup granularity type of the remaining data blocks is uniformly determined according to the average distribution entropy value within the cluster.

[0095] For example, for each modified data block, the metadata management service returns the distribution information of all replicas for that data block. This information includes the total number of replicas, the identifier of each storage node where a replica resides, and the storage path of the replica on each storage node. The storage node identifier is represented by a unique string, formatted as a combination of the node's IP address and port number. The number of replicas distributed on each storage node is counted, using the storage node as a classification dimension and the number of replicas as a frequency value to construct a replica distribution frequency table. This frequency table is stored in a hash mapping structure, with the storage node identifier as the key and the number of replicas on that node as the value. For example, data block BLOCK_A has 5 replicas, distributed across storage nodes NODE_192_168_1_10_9000, NODE_192_168_1_11_9000, NODE_192_168_1_12_9000, NODE_192_168_1_10_9000, and NODE_192_168_1_11_9000. Statistics show that there are 2 replicas on NODE_192_168_1_10_9000, 2 replicas on NODE_192_168_1_11_9000, and 1 replica on NODE_192_168_1_12_9000. The replica distribution frequency table contains 3 key-value pairs: (NODE_192_168_1_10_9000, 2), (NODE_192_168_1_11_9000, 2), and (NODE_192_168_1_12_9000, 1).

[0096] Iterate through all key-value pairs in the replica distribution frequency table, extract the number of replicas on each storage node, and divide the number of replicas by the total number of replicas to obtain the replica distribution ratio of that storage node. Calculate the logarithm of the replica distribution ratio for each storage node, with the base of the logarithm being 2, and round the result to four decimal places. Multiply the replica distribution ratio of each storage node by the negative of its logarithm to obtain the entropy contribution value of that storage node. Sum the entropy contributions of all storage nodes to obtain the distribution entropy value of the data block, rounding it to three decimal places. When replicas are completely concentrated on a single storage node, the distribution entropy value is 0. When replicas are evenly distributed across multiple storage nodes, the distribution entropy value reaches its maximum value, which is equal to the logarithm of the number of storage nodes.

[0097] The data block dependency graph is stored in adjacency list format. Each data block node maintains an incoming edge list, which records the identifiers of all source data blocks pointing to that data block and their edge weights. For each changed data block, its node record in the data block dependency graph is queried, and the number of edges in the incoming edge list is counted as the in-degree value. The in-degree value is a non-negative integer, ranging from 0 to the total number of data blocks minus 1. When the in-degree value is greater than a preset in-degree threshold, the data block is identified as a critical path data block. The preset in-degree threshold is set to 3 by default. This threshold is determined based on the statistical distribution of data block dependencies in historical backup cycles, covering approximately 85% of ordinary data blocks.

[0098] The distribution threshold is dynamically adjusted based on the in-degree value and the determination result of critical path data blocks. The baseline value of the distribution threshold is set to 1.5, which corresponds to the entropy level of replicas evenly distributed across approximately 3 storage nodes. When the in-degree value of a data block does not exceed the preset in-degree threshold, the distribution threshold remains unchanged at the baseline value of 1.5. When a data block is a critical path data block, the distribution threshold is reduced by multiplying the baseline value by 0.2. The reduced distribution threshold is the baseline value minus the reduction amount. The adjusted distribution threshold is 1.5 - 1.5 × 0.2 = 1.2. The purpose of reducing the distribution threshold is to increase the probability of critical path data blocks using block-level backups, ensuring that data blocks that have a significant impact on data flow propagation receive finer-grained backup protection. The minimum value of the distribution threshold is limited to 0.5. When the calculated result is below 0.5, it is truncated to 0.5 to avoid excessive use of block-level backups due to an excessively low threshold, which would increase the load on the backup system. For example, data block BLOCK_B is a critical path data block with a baseline distribution threshold of 1.5, a reduction of 0.3, and an adjusted distribution threshold of 1.2.

[0099] The backup granularity type is determined by comparing the distribution entropy value with an adjusted distribution threshold. When the distribution entropy value of a data block is higher than the adjusted distribution threshold, the replicas of that data block are considered relatively dispersed, and a block-level backup granularity type is used. Block-level backup uses a single data block as the operation unit; the backup system only transmits the actual data content of that data block, and the amount of data transmitted is equal to the data block size. When the distribution entropy value of a data block is lower than the adjusted distribution threshold, the replicas of that data block are considered relatively concentrated, and a file-level backup granularity type is used. File-level backup uses the complete file to which the data block belongs as the operation unit; the backup system transmits all data blocks of that file, and the amount of data transmitted is equal to the total file size. The result of the backup granularity type determination is stored in a backup granularity mapping table. The mapping table is stored in a hash mapping structure, with the key being the data block identifier and the value being the backup granularity type enumeration value, which includes two types: BLOCK_LEVEL and FILE_LEVEL.

[0100] The variance calculation of distribution entropy values ​​within independent data block clusters is performed for each identified data block cluster. The distribution entropy values ​​of all data blocks within the cluster are extracted, and their arithmetic mean is calculated as the cluster's distribution entropy mean. For each data block within the cluster, the difference between its distribution entropy value and the cluster's distribution entropy mean is calculated, and the squared differences are accumulated. The accumulated result is divided by the number of data blocks in the cluster to obtain the distribution entropy variance, which is rounded to four decimal places. A preset difference threshold is set to 0.25, corresponding to a standard deviation of approximately 0.5 for the distribution entropy values. When the distribution entropy variance exceeds the preset difference threshold, abnormal data blocks whose distribution entropy values ​​deviate from the cluster's mean are identified. The deviation criterion is to first calculate the difference between the data block's distribution entropy value and the cluster's distribution entropy mean, then take the absolute value of the difference and determine if it is greater than the square root of the distribution entropy variance. If it is, a deviation is indicated. Data blocks meeting the deviation criteria are marked as abnormal data blocks, and their backup granularity type is forcibly adjusted to block-level backup in the backup granularity mapping table. For the remaining data blocks within the cluster that are not marked as abnormal, the backup granularity type is uniformly determined by comparing the average distribution entropy within the cluster with the adjusted distribution threshold. When the average distribution entropy within the cluster is higher than the distribution threshold, all non-abnormal data blocks within the cluster are backed up at the data block level; when the average distribution entropy within the cluster is lower than the distribution threshold, all non-abnormal data blocks within the cluster are backed up at the file level.

[0101] This invention dynamically determines the backup granularity type based on the distribution entropy value and performs differentiated processing on abnormal data blocks, realizing adaptive adjustment of the backup strategy. This not only ensures the backup quality of critical data but also optimizes the utilization efficiency of system resources and improves the overall performance of backup.

[0102] In one optional implementation, the steps of generating a backup execution sequence according to the topology sorting result, prioritizing the backup of the critical path data blocks and backing up the independent data block clusters in parallel, and simultaneously performing backups according to the backup granularity type, and generating backup metadata containing dependency identifiers and distribution entropy values ​​include:

[0103] Based on the topology sorting results, a hierarchical backup queue is constructed, placing the critical path data blocks in the priority level and the independent data block clusters in the parallel level; the backup resource ratio is dynamically calculated based on the number of critical path data blocks and the sum of their dependencies, and corresponding system I / O bandwidth and storage resources are allocated to the priority level to generate a backup execution sequence;

[0104] Extract the backup granularity type of the current data block to be backed up from the backup execution sequence. When the backup granularity type is block-level backup, perform backup operations on all copies. When the backup granularity type is file-level backup, perform backup operations only on the primary copy.

[0105] During the backup operation, dependency direction and dependency strength are extracted from the data block dependency graph. The dependency direction is converted into a dependency relationship identifier, and a backup metadata record containing the dependency relationship identifier, dependency strength, and distribution entropy value is created.

[0106] Once the priority-level backup is complete, initiate the parallel-level backup; monitor the backup progress, and if the backup time for a data block cluster exceeds the predicted time threshold, move its incomplete data blocks to the head of the queue and increase the number of threads.

[0107] A metadata index tree is constructed based on the dependency relationship identifier, and repeated dependency patterns are compressed and encoded to generate backup metadata files.

[0108] For example, the hierarchical backup queue is constructed based on topology sorting results and data block type identifiers. The backup scheduling module maintains a two-level queue structure, consisting of two independent queues: a priority-level queue and a parallel-level queue. The priority-level queue is implemented as a first-in-first-out linear queue, with each element being an identifier for a single critical path data block. The data block sequence in the topology sorting result is traversed, and the type identifier of each data block is queried. When a data block is marked as a critical path data block, its identifier is inserted at the tail of the priority-level queue. The parallel-level queue is implemented using a nested structure, with the outer layer being a list of data block clusters and the inner layer being a set of data block identifiers contained in each cluster. The identified independent data block clusters are traversed, and each cluster is inserted as an independent element into the parallel-level queue.

[0109] The number of data blocks in the priority queue is taken as the number of critical path data blocks. All incoming edges of critical path data blocks in the data block dependency graph are traversed, and the edge weight of each incoming edge is accumulated to obtain the dependency strength of that data block. The dependency strengths of all critical path data blocks are summed to obtain the total dependency strength. The backup resource ratio is calculated as (number of critical path data blocks × 10 + total dependency strength × 5) ÷ 100, with the result rounded to two decimal places. The backup resource ratio is limited to the range of 0.3 to 0.7. Values ​​less than 0.3 are truncated to 0.3, and values ​​greater than 0.7 are truncated to 0.7. The I / O bandwidth allocated for the priority tier is the total system I / O bandwidth multiplied by the backup resource ratio, rounded down to the nearest multiple of 10 megabytes per second. The total storage resources are the available space of the backup target storage pool. The storage resources allocated for the priority tier are the total storage resources multiplied by the backup resource ratio, rounded down to the nearest multiple of 1 gigabyte.

[0110] The backup execution sequence adopts a two-segment structure. The first segment contains all elements of the priority queue, and the second segment contains all elements of the parallel queue. Each element in the sequence is appended with an execution strategy identifier: the execution strategy identifier for priority elements is SEQUENTIAL, and the execution strategy identifier for parallel elements is PARALLEL. The backup execution sequence is stored in JSON format. The JSON object contains a `sequence` field, which is an array type. The array elements contain a `block_id` field storing the data block identifier, a `strategy` field storing the execution strategy identifier, and a `resource_quota` field storing resource quota information. The resource quota information includes `io_bandwidth` and `storage_space` fields, corresponding to the allocated I / O bandwidth and storage space, respectively.

[0111] The backup granularity type is extracted by querying the backup granularity mapping table. The backup execution module traverses the data blocks to be backed up in the order of the backup execution sequence. For each data block, the corresponding backup granularity type enumeration value is queried in the backup granularity mapping table using the data block identifier as the key. When the query result is BLOCK_LEVEL, the backup execution module obtains the complete list of replicas for that data block from the replica distribution location information and performs a backup operation on each replica in the list. The backup operation reads replica data by calling the storage system's data read interface and writes the read data to the pre-allocated storage block of the backup target storage pool. When the query result is FILE_LEVEL, the backup execution module extracts the file identifier belonging to the data block from the metadata record, queries the primary replica location of that file, and performs a backup operation only on the primary replica. The rule for determining the primary replica is to select the first replica in the replica list as the primary replica. This rule is pre-set by the distributed file system when the replica is created.

[0112] The backup execution module initiates a metadata collection thread for each data block to be backed up. This thread queries all incoming and outgoing edges of the data block in its dependency graph. The list of source node identifiers for incoming edges is converted into dependency identifiers of type DEPENDS_ON, and the list of target node identifiers for outgoing edges is converted into dependency identifiers of type DEPENDED_BY. The weight value of each edge is stored as dependency strength in the metadata record. The distribution entropy value is directly obtained from the previously calculated distribution entropy value. The backup metadata record is stored in a structured format, containing a block_id field storing the data block identifier, a depends_on field storing an array of dependencies of type DEPENDS_ON, a depends_by field storing an array of dependencies of type DEPENDED_BY, a dependency_strength field storing a dependency strength mapping table, and a distribution_entropy field storing the distribution entropy value.

[0113] The backup scheduling module listens for empty queue events in the priority queue. Once all data block backup operations in the priority queue are completed and removed from the queue, a parallel hierarchical backup start signal is triggered. Parallel hierarchical backup allocates an independent backup thread pool to each data block cluster in the parallel hierarchical queue. The default thread pool size is 4, configurable from 2 to 16. Threads within each thread pool concurrently process data block backup tasks within the cluster. Each thread retrieves the identifier of the data block to be backed up from the cluster's data block set, performs the backup operation, and updates the backup progress counter. The backup progress monitoring module polls the backup progress of each data block cluster every 1 second, recording the ratio of completed data blocks to the total number of data blocks. The prediction time threshold is calculated based on the average size of the data blocks within the cluster and the allocated I / O bandwidth. The formula is: total data block size divided by allocated I / O bandwidth multiplied by 1.5, with the result rounded up to the nearest second. When the actual backup time of a data block cluster exceeds the predicted time threshold, the backup scheduling module extracts the identifiers of the data blocks in the cluster that have not yet been backed up, removes these identifiers from the data block set in the cluster, inserts them at the head of the parallel hierarchical queue, and increases the thread pool size of the cluster by 2 threads, with the maximum thread pool size after the increase being 16.

[0114] The backup metadata management module collects all backup metadata records, extracts the `depends_on` and `depended_by` fields from each record, and constructs a directed graph structure. The directed graph is stored in adjacency list format, with nodes representing data block identifiers and edges representing dependencies. The directed graph is traversed to identify repeating dependency patterns, defined as multiple data blocks having the same `depends_on` array or the same `depended_by` array. Repeating dependency patterns are compressed and encoded, and a unique pattern identifier is assigned to each unique dependency pattern. Pattern identifiers are incrementing integers, starting from 1. The `depends_on` and `depended_by` fields in the backup metadata records are replaced with the pattern identifiers. The original dependency arrays are stored in a pattern mapping table, which is stored in a hash mapping structure, with the pattern identifier as the key and the dependency array as the value. The metadata index tree is implemented using a B-tree structure, with the data block identifier as the key and the storage offset of the backup metadata record as the value. The backup metadata file contains three parts: serialized metadata index tree data, serialized pattern mapping table data, and serialized backup metadata record data, separated by fixed-length delimiters. The backup metadata file is stored in binary format. The file header includes a version number, index tree offset, pattern mapping table offset, and metadata record offset fields. For example, data blocks BLOCK_R and BLOCK_S both depend on BLOCK_U and BLOCK_V, with a repeated dependency pattern of [BLOCK_U, BLOCK_V], and a pattern identifier of 1 is assigned. The depends_on field of BLOCK_R is replaced with 1, and the depends_on field of BLOCK_S is replaced with 1. The pattern mapping table stores {1:[BLOCK_U, BLOCK_V]}.

[0115] This invention achieves efficient scheduling and management of the backup process by constructing a hierarchical backup queue and dynamically allocating system resources, combined with metadata compression encoding and index tree optimization. At the same time, it ensures priority backup of critical data, thereby improving the reliability and efficiency of the backup process.

[0116] In one optional implementation, the steps of performing dependency consistency verification after recovery based on the dependency identifiers in the backup metadata, and adaptively adjusting the dependency strength in the data block dependency graph using the distribution entropy value change trend over multiple backup cycles, include:

[0117] After data recovery is complete, the dependency identifiers of all data blocks are extracted from the backup metadata to construct a dependency verification graph of the recovered data blocks;

[0118] Traverse each dependency edge in the data block dependency verification graph, and detect the actual read / write access relationship between the predecessor and successor data blocks connected by the dependency edge in the recovered file system. When the actual read / write access relationship is inconsistent with the dependency direction indicated by the dependency relationship identifier, it is marked as a dependency consistency anomaly.

[0119] For data blocks marked as having dependency consistency anomalies, the distribution entropy value sequence of the data block in the current backup cycle and historical backup cycles is extracted from the backup metadata, and the change trend coefficient of the distribution entropy value sequence is calculated; based on the inverse correlation between the change trend coefficient and the direction of change of the distribution entropy value, the dependency strength of the corresponding dependency edge in the dependency graph of the data block is adjusted;

[0120] The adjusted dependency strength is updated to the data block dependency graph for topology sorting and critical path data block identification in the next backup cycle.

[0121] For example, the dependency consistency verification module is automatically triggered after the data recovery operation is completed, reading the schema mapping table and backup metadata records from the backup metadata file. The backup metadata records contain the `block_id`, `depends_on`, and `depended_by` fields, where the `depends_on` and `depended_by` fields store schema identifiers. The verification module iterates through all backup metadata records, using the schema identifiers to query the corresponding dependency array in the schema mapping table. The dependency array contains a list of predecessor data block identifiers or a list of successor data block identifiers. The data block dependency verification graph is stored using a directed graph data structure, where graph nodes are data block identifiers, and directed edges represent dependencies, with the edges pointing from the predecessor data block to the successor data block. For each backup metadata record, a directed edge is established between each predecessor data block in the dependency array corresponding to the `depends_on` field and the current data block. The edge weight is initialized to the dependency strength value of the corresponding predecessor data block in the `dependency_strength` field of the backup metadata record. The verification graph is stored in memory in the form of an adjacency list. The key of the adjacency list is the data block identifier, and the value is a list of all outgoing edges of the data block. Each outgoing edge contains two fields: the target node identifier and the edge weight.

[0122] Actual read / write access relationships are detected through file system access log analysis. After recovery, the file system enters access monitoring mode after data recovery is complete. The file system kernel module intercepts all data block-level read / write operations, recording the operation type, source data block identifier, target data block identifier, and timestamp. Access logs are stored in time-series format. Each log record contains an `operation_type` field storing a READ or WRITE enumeration value, a `source_block` field storing the identifier of the data block initiating the access, a `target_block` field storing the identifier of the accessed data block, and a `timestamp` field storing the UNIX timestamp of the operation time. The dependency consistency verification module sets the access monitoring window period, with a default window period of 600 seconds, configurable from 300 to 1800 seconds. It traverses each directed edge in the data block dependency verification graph, extracting the source node identifier as the predecessor data block and the target node identifier as the successor data block. It queries all read / write records in the access log involving predecessor and successor data blocks, filtering records that satisfy the `timestamp` field being within the monitoring window period. Access relationship analysis is performed on the filtered records. Records where the `source_block` field equals the successor data block and the `target_block` field equals the predecessor data block are marked as reverse access. The dependency direction determination rule is that the dependency direction indicated by the dependency relationship identifier should be that the predecessor data block is accessed by the successor data block, i.e., the successor data block reads data from the predecessor data block. When a reverse access is detected or the access log does not contain a record of a successor data block accessing a predecessor data block, the actual read / write access relationship is determined to be inconsistent with the dependency direction indicated by the dependency relationship identifier. This directed edge is marked as a dependency consistency anomaly, and the anomaly flag is stored in the edge's attribute field `consistency_status`, with an enumeration value of `ANOMALY`.

[0123] The backup system maintains a backup metadata repository. Backup metadata files generated after each backup cycle are stored in the repository, named with a timestamp. The dependency consistency verification module extracts the identifier of each data block marked as having dependency consistency issues and queries the backup metadata repository in reverse chronological order for historical backup metadata files containing that data block. The query depth defaults to the last 5 backup cycles, but can be configured from 3 to 10 backup cycles. For each retrieved backup metadata file, the system parses the backup metadata record of the corresponding data block and extracts the value of the `distribution_entropy` field. The distribution entropy value sequence is arranged chronologically by backup cycle. Each element in the sequence contains a `backup_cycle_id` field storing the backup cycle identifier and an `entropy_value` field storing the distribution entropy value. The backup cycle identifier is an incrementing integer, starting from 1, with the identifier for the current backup cycle being the maximum value.

[0124] The trend coefficient is calculated using the linear regression slope method. The dependency strength adjustment module performs linear fitting on the distribution entropy value sequence, using the backup period identifier as the independent variable and the distribution entropy value as the dependent variable. The slope of the fitted line is calculated using the least squares method, and the slope value is the trend coefficient. The calculation process involves summing the products of the backup period identifier and the distribution entropy value for all elements in the sequence, denoted as SUM_XY. Then, the sum of the backup period identifiers is calculated, denoted as SUM_X. Finally, the sum of the distribution entropy values ​​is calculated, denoted as SUM_Y. The sum of the squares of the backup period identifiers is calculated, denoted as SUM_XX. The sequence length is denoted as N. The trend coefficient = (N×SUM_XY - SUM_X×SUM_Y) / (N×SUM_XX - SUM_X) 2 The calculation results are rounded to four decimal places. A positive trend coefficient indicates that the distribution entropy value is increasing, while a negative value indicates that the distribution entropy value is decreasing. The absolute value indicates the strength of the trend.

[0125] The reverse association relationship is defined as follows: when the distribution entropy value is increasing (i.e., the trend coefficient is positive), the dependency strength of the corresponding dependency edge should decrease; when the distribution entropy value is decreasing (i.e., the trend coefficient is negative), the dependency strength should increase. The dependency strength adjustment is calculated by multiplying the absolute value of the trend coefficient by an adjustment factor. The default value of the adjustment factor is 50, and the configurable range is 20 to 100. The adjusted dependency strength = original dependency strength - trend coefficient × adjustment factor, and the result is rounded down. The dependency strength value is limited to between 1 and 20. When the adjusted dependency strength is less than 1, it is truncated to 1; when the adjusted dependency strength is greater than 20, it is truncated to 20. The dependency strength adjustment module traverses all directed edges marked as dependency consistency anomalies in the verification graph, extracts the identifiers of the predecessor and successor data blocks connected by the edge, locates the corresponding dependency edge in the data block dependency graph, and updates the weight field of the edge with the adjusted dependency strength.

[0126] The updated dependency strength is achieved through the data block dependency graph persistence interface. The data block dependency graph management module maintains the synchronization mechanism between the graph's memory structure and disk storage, and calls the `update_edge_weight` interface to update the weight value of the specified dependency edge. Interface parameters include the `source_block` field storing the predecessor data block identifier, the `target_block` field storing the successor data block identifier, and the `new_weight` field storing the adjusted dependency strength value. When the interface is executed, the corresponding edge is located in the in-memory adjacency list structure, the edge weight field is modified, and the modification operation is appended to the change log file. The change log file is opened in append-only mode, and each change record contains the operation type, the edge's source and target identifiers, the old weight value, the new weight value, and a timestamp. Graph persistence is triggered when the change log file size exceeds 10 megabytes or when the last persistence time exceeds 300 seconds. After triggering, the in-memory adjacency list structure is serialized into JSON format and written to the disk storage file, and the change log file is cleared. The updated data block dependency graph is used in the topology sorting module and critical path data block identification module for the next backup cycle. Both modules load the latest graph data from the disk storage file at the start of the backup cycle.

[0127] This invention achieves dynamic optimization of backup and recovery quality by verifying the dependency consistency of the recovered data and adaptively adjusting the dependency strength, ensuring the accuracy of data recovery. Furthermore, it continuously optimizes the data block dependency graph through a feedback mechanism, thereby improving the accuracy of subsequent backups.

[0128] A second aspect of the present invention provides an electronic device, comprising:

[0129] processor;

[0130] Memory used to store processor-executable instructions;

[0131] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0132] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0133] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Hadoop cluster incremental backup and recovery method based on data block optimization, characterized in that, include: Obtain the distribution status information and historical change records of data blocks in the distributed file system, extract the spatiotemporal correlation features between data blocks, and construct a data block dependency graph that includes dependency direction and dependency strength; Identify the changed data blocks, perform topological sorting of the changed data blocks according to the data block dependency graph, and identify critical path data blocks and clusters of independent data blocks; The distribution entropy value is calculated based on the replica distribution location of the changed data block, and the backup granularity type is determined based on the distribution entropy value. Specifically, this includes: obtaining the complete replica distribution location of each changed data block in the distributed file system; calculating the distribution entropy value of the changed data block based on the number of replicas distributed on each storage node; extracting the in-degree value of the changed data block from the data block dependency graph; dynamically adjusting the distribution threshold based on the in-degree value, wherein when the in-degree value indicates that the data block is a critical path data block, the distribution threshold is lowered; when the distribution entropy value is higher than the adjusted distribution threshold, the backup granularity type is determined to be block-level backup; when the distribution entropy value is lower than the adjusted distribution threshold, the backup granularity type is determined to be file-level backup; for the independent data block clusters, the variance of the distribution entropy value of each data block within the cluster is calculated; when the variance of the distribution entropy value exceeds a preset difference threshold, abnormal data blocks whose distribution entropy value deviates from the mean are identified, and the backup granularity type of the abnormal data blocks is adjusted to block-level backup; the backup granularity type of the remaining data blocks is uniformly determined based on the average distribution entropy value within the cluster. A backup execution sequence is generated based on the topological sorting result. The critical path data blocks are backed up first, and the independent data block clusters are backed up in parallel. At the same time, backup is performed according to the backup granularity type, and backup metadata containing dependency identifiers and distribution entropy values ​​is generated. Dependency consistency is verified after recovery based on the dependency identifiers in the backup metadata, and the dependency strength in the data block dependency graph is adaptively adjusted using the distribution entropy value change trend of multiple backup cycles.

2. The method according to claim 1, characterized in that, The steps for obtaining the distribution status information and historical change records of data blocks in a distributed file system, extracting the spatiotemporal correlation features between data blocks, and constructing a data block dependency graph containing dependency direction and dependency strength include: Based on the distribution status information, the physical distribution location relationship and logical read / write access relationship of the data blocks are extracted as spatial association features; based on the historical change records, the change time sequence and change propagation path of the data blocks are extracted as temporal association features. By coupling the spatial correlation features with the temporal correlation features, we can identify pairs of correlated data blocks that exhibit temporal correlation changes and spatial dependencies. The dependency direction is determined based on the order of change time and read / write access relationship of the first and second data blocks in the associated data block pair. The dependency strength is determined based on the frequency and stability of the associated data block pair in multiple backup cycles. A data block dependency graph is constructed, where nodes represent data blocks, directed edges represent the dependency direction, and edge weights represent the dependency strength.

3. The method according to claim 2, characterized in that, The steps of coupling the spatial correlation features with the temporal correlation features to identify correlated data block pairs that have temporal correlation changes and spatial dependencies include: For each data block, a spatial proximity matrix is ​​constructed based on the physical distribution location relationship, and a read-write dependency matrix is ​​constructed based on the logical read-write access relationship; the data block pairs that actually underwent association changes in historical backup cycles are statistically analyzed, and the contribution ratio of spatial factors and read-write factors to association changes is calculated. Based on the contribution ratio, the fusion weight of the spatial proximity matrix and the read-write dependency matrix is ​​dynamically determined to generate a spatial association strength matrix; Based on the change time series and change propagation path, a change causal chain is extracted. An adaptive time window threshold is calculated by analyzing the historical change propagation speed of data blocks within the change causal chain. Data block pairs with change time intervals less than the adaptive time window threshold and located on the same change causal chain are marked as time-related data block pairs. The data block pairs with a correlation strength exceeding a preset spatial threshold in the spatial correlation strength matrix are intersected with the time-related data block pairs. The intersection results are then subjected to multi-period stability verification. The frequency of occurrence of the data block pairs in multiple consecutive backup periods is counted. When the frequency of occurrence exceeds a preset stability threshold, the data block pairs are confirmed as related data block pairs.

4. The method according to claim 1, characterized in that, The steps of identifying modified data blocks, performing topological sorting of the modified data blocks according to the data block dependency graph, and identifying critical path data blocks and independent data block clusters include: The distribution state information is compared with the pre-stored baseline distribution state information. When there is a difference between the current state characteristics of a data block and the baseline state characteristics, the corresponding data block is marked as a changed data block. Based on the directed edges in the data block dependency graph, calculate the in-degree and out-degree values ​​of each changed data block; mark changed data blocks with in-degree values ​​greater than a preset in-degree threshold or out-degree values ​​greater than a preset out-degree threshold as critical path data blocks; The modified data blocks are topologically sorted in the data block dependency graph to obtain a sequence of data blocks that satisfy the dependency relationship; a set of data blocks that are not connected by directed edges to each other is identified in the data block sequence, and the data block set is determined as a data block cluster that is independent of each other; the data blocks in the data block cluster are sorted in descending order according to their global centrality index in the data block dependency graph, and backed up in sequence.

5. The method according to claim 1, characterized in that, The steps of generating a backup execution sequence according to the topology sorting result, prioritizing the backup of critical path data blocks and backing up independent data block clusters in parallel, and performing backups according to the backup granularity type, and generating backup metadata containing dependency identifiers and distribution entropy values ​​include: Based on the topology sorting results, a hierarchical backup queue is constructed, placing the critical path data blocks in the priority level and the independent data block clusters in the parallel level; the backup resource ratio is dynamically calculated based on the number of critical path data blocks and the sum of their dependencies, and corresponding system I / O bandwidth and storage resources are allocated to the priority level to generate a backup execution sequence; Extract the backup granularity type of the current data block to be backed up from the backup execution sequence. When the backup granularity type is block-level backup, perform backup operations on all copies. When the backup granularity type is file-level backup, perform backup operations only on the primary copy. During the backup operation, dependency direction and dependency strength are extracted from the data block dependency graph. The dependency direction is converted into a dependency relationship identifier, and a backup metadata record containing the dependency relationship identifier, dependency strength, and distribution entropy value is created. Once the priority-level backup is complete, initiate the parallel-level backup; monitor the backup progress, and if the backup time for a data block cluster exceeds the predicted time threshold, move its incomplete data blocks to the head of the queue and increase the number of threads. A metadata index tree is constructed based on the dependency relationship identifier, and repeated dependency patterns are compressed and encoded to generate backup metadata files.

6. The method according to claim 1, characterized in that, The steps of performing dependency consistency verification after recovery based on dependency identifiers in the backup metadata, and adaptively adjusting the dependency strength in the data block dependency graph using the distribution entropy value change trend over multiple backup cycles, include: After data recovery is complete, the dependency identifiers of all data blocks are extracted from the backup metadata to construct a dependency verification graph of the recovered data blocks; Traverse each dependency edge in the data block dependency verification graph, and detect the actual read / write access relationship between the predecessor and successor data blocks connected by the dependency edge in the recovered file system. When the actual read / write access relationship is inconsistent with the dependency direction indicated by the dependency relationship identifier, it is marked as a dependency consistency anomaly. For data blocks marked as having dependency consistency anomalies, the distribution entropy value sequence of the data block in the current backup cycle and historical backup cycles is extracted from the backup metadata, and the change trend coefficient of the distribution entropy value sequence is calculated; based on the inverse correlation between the change trend coefficient and the direction of change of the distribution entropy value, the dependency strength of the corresponding dependency edge in the dependency graph of the data block is adjusted; The adjusted dependency strength is updated to the data block dependency graph for topology sorting and critical path data block identification in the next backup cycle.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images