Method for backup and recovery management based on multiple data

Abstract

The present application relates to the technical field of data management, and particularly relates to a backup and recovery management method based on multiple data, which comprises the following steps: real-time monitoring of update frequency and associated dependency relationship of multiple data streams, generating an activity atlas comprising time sensitivity weight and cross-data chain impact factor; dynamically dividing a time window according to the activity atlas, performing atomized snapshot backup on the associated data chain when the time window is closed, and generating a space-time consistency identifier; reversely analyzing the topology structure of the associated data chain based on the space-time consistency identifier, preferentially recovering high-impact-factor nodes and parallelly rebuilding dependent paths. The present application effectively avoids the blocking and failure caused by dependency conflicts in the traditional recovery process, is particularly suitable for business systems with complex data chain cross, has high reliability and recovery robustness, and significantly shortens the system recovery time and business interruption duration.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and in particular to a method for backup and recovery management based on multiple data sets. Background Technology

[0002] With the widespread deployment of distributed systems and multi-source heterogeneous data architectures in scenarios such as cloud computing, the Internet of Things, and smart manufacturing, data backup and recovery have become core means to ensure system reliability and business continuity. Existing backup technologies are mostly based on fixed-period triggering, single-point snapshots, or simple file-level copying, lacking awareness of data flow status and structural dependencies, and are difficult to adapt to data scenarios with high-frequency updates, strong coupling, and multi-chain parallelism.

[0003] On the one hand, current systems often fail to recognize differences in data activity levels, leading to the use of the same backup strategy for high-frequency critical data and low-frequency peripheral data, resulting in wasted resources or backup delays. On the other hand, when faced with data chain structures involving cross-service calls, nested dependencies, or event chain coupling, traditional backup mechanisms cannot identify dependency topologies, easily leading to state misalignment, data loss, or circular dependency blockages during the recovery process.

[0004] Furthermore, most recovery processes employ linear scanning or reverse timestamp restoration methods, failing to adequately consider data dependencies and priorities, and lacking efficient concurrency scheduling and exception handling mechanisms. Especially in the event of system-level failures or link interruptions, traditional methods are inefficient and risky when rebuilding complex business data chains, easily leading to prolonged business service interruptions. Summary of the Invention

[0005] This invention provides a backup and recovery management method based on multiple data sets, which has the capabilities of data activity identification, dependency structure awareness and efficient recovery scheduling, so as to achieve a more real-time and consistent end-to-end data protection system.

[0006] The backup and recovery management method based on multiple data sets includes the following steps:

[0007] S1. Dynamic Data Activity Awareness: Real-time monitoring of the update frequency and correlation dependencies of multiple data streams, generating an activity map including time-sensitive weights and cross-data-chain influence factors;

[0008] S2. Spatiotemporal Consistency Backup: Dynamically divide time windows based on the activity spectrum, and perform atomic snapshot backups on the associated data chains when the time window closes to generate a spatiotemporal consistency identifier;

[0009] S3. Topology-driven recovery: Based on the spatiotemporal consistency identifier, reverse the topology of the associated data chain, prioritize the recovery of nodes with high impact factors and rebuild the dependency path in parallel.

[0010] Optionally, S1 includes quantizing the update frequency, specifically including creating a sliding time window for each data stream, counting the number of data updates within the sliding time window and adding a time decay factor to generate an update frequency value.

[0011] Optionally, S1 further includes parsing the associated dependencies, specifically including extracting the call chain identifiers embedded in the data stream, constructing a dependency graph across the data stream, and calculating the betweenness centrality of each node in the dependency graph as an initial influence factor.

[0012] Optionally, the update frequency value is normalized to generate a time-sensitive weight, and the cross-data chain influence factor is iteratively corrected based on the betweenness centrality and dependency depth in the dependency graph to finally generate an activity graph.

[0013] Optionally, the modified cross-datalink impact factor is expressed as follows: Where, β v C is the revised cross-datalink impact factor. B (v) represents the betweenness centrality value of node v in the dependency graph. Indicates the dependency depth penalty factor term, d v γ represents the dependency depth, which is the shortest path length from the node to the source node, and γ represents the dependency depth penalty factor.

[0014] Optionally, generating the activity map specifically includes normalizing the update frequency value of the data corresponding to each node to obtain a time sensitivity weight at a unified scale, characterizing the activity level of the data in the current time period. Combining the betweenness centrality of each node in the dependency graph with its dependency depth in the topology, a dependency depth correction factor is applied to the centrality index to obtain the initial cross-data chain influence factor of each node, suppressing the influence weight of deep indirect dependent nodes in the global topology. When the time sensitivity weight of a node exceeds a preset threshold, a dynamic frequency coupling mechanism is activated to adjust the gain of the cross-data chain influence factor of its directly downstream dependent nodes, strengthening the driving effect of highly active nodes on downstream nodes in its dependency path. The time sensitivity weight of each node is combined with the corrected cross-data chain influence factor to form an activity map that supports the spatiotemporal consistency backup and topology-driven recovery strategy.

[0015] Optionally, S2 specifically includes:

[0016] S21, Window dynamic calculation: Determine the length of the backup time window for each data link based on the time sensitivity weight value in the activity map;

[0017] S22, Atomic Snapshot Trigger: When an update event is detected in any node of the associated data chain, the backup time window countdown of the chain is reset. When the backup time window is closed, the state of all associated nodes is frozen and a consistent snapshot is generated. Here, the associated data chain represents a set of nodes with direct or indirect dependencies constructed through a dependency graph.

[0018] S23, Identifier Encoding: The snapshot timestamp, data chain topology hash value, and check code are concatenated according to a preset format to generate a spatiotemporal consistency identifier, wherein the topology hash value comes from the cross-data chain influence factor distribution in the activity graph.

[0019] Optionally, the length of each data link backup time window is calculated as follows: Among them, W i T represents the actual backup time window length of data chain i. base α is the set base time window duration. i Let i be the time-sensitivity weight of data chain i. This is the dynamic adjustment coefficient.

[0020] Optionally, S3 specifically includes:

[0021] S31, Topology Reconstruction: Parse the topology hash value in the spatiotemporal consistency identifier to restore the node dependencies and cross-data chain influence factors recorded in the live graph;

[0022] S32, Recovery Priority Sorting: Nodes are sorted in descending order of cross-datalink impact factor to generate the main recovery queue, while nodes without predecessor dependencies in the topology are identified to form parallel recovery sub-queues;

[0023] S33, phased recovery.

[0024] Optionally, the phased recovery process includes three phases, wherein:

[0025] Phase 1: Extract high-impact factor nodes above the preset threshold from the head of the main recovery queue and simultaneously start the preloading of all their dependencies;

[0026] Phase 2: Initiate multi-threaded concurrent recovery of nodes in the parallel recovery sub-queue, while monitoring the dependency satisfaction status of nodes in the main queue;

[0027] The third stage: When a circular dependency is detected, the loop is forcibly unlooped based on the influence factor weights and conflict markers are recorded.

[0028] The beneficial effects of this invention are:

[0029] This invention introduces a dynamic update frequency awareness mechanism, combined with call chain parsing and graph structure analysis, to construct an activity graph that reflects the activity and dependencies of data updates. Based on time sensitivity weights, the backup time window length is controlled, enabling high-frequency update data chains to obtain shorter backup cycles, while low-frequency data is adaptively extended. This improves resource utilization and the accuracy of backup strategies, and compared with traditional fixed-time-cycle backup methods, it has stronger real-time performance and differentiated scheduling capabilities.

[0030] This invention achieves a backup consistency scheme with structure recognition capabilities by combining cross-node state freezing and atomic snapshot mechanisms with topological hashing and a unified version number binding strategy. Compared with traditional snapshot methods centered on time points, this invention integrates the dependency structure and distribution of influencing factors in the activity graph during snapshot generation, ensuring that not only is time consistent during data recovery, but the data chain structure can also be reconstructed according to the dependency relationship, thereby improving the data consistency and integrity assurance capabilities in multi-data-chain and strongly coupled scenarios.

[0031] This invention constructs a main recovery queue based on the priority of impact factors and is supplemented by a parallel sub-queue scheduling strategy. This enables concurrent recovery of nodes without dependencies and priority reconstruction of core nodes. Through circular dependency detection and intelligent loop unblocking mechanism, it effectively avoids blocking and failure caused by dependency conflicts in the traditional recovery process. It is especially suitable for business systems with complex data chain intersections, has high reliability and recovery robustness, and significantly shortens system recovery time and business interruption duration. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of spatiotemporal consistency backup according to an embodiment of the present invention. Detailed Implementation

[0035] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0036] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0037] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0038] like Figures 1-2 As shown, the backup and recovery management method based on multiple data sets includes the following steps:

[0039] S1. Dynamic Data Activity Awareness: Real-time monitoring of the update frequency and correlation dependencies of multiple data streams, generating an activity map including time-sensitive weights and cross-data-chain influence factors;

[0040] S2. Spatiotemporal Consistency Backup: Dynamically divide time windows based on the activity spectrum, and perform atomic snapshot backups on the associated data chains when the time window closes to generate a spatiotemporal consistency identifier;

[0041] S3. Topology-driven recovery: Based on the spatiotemporal consistency identifier, reverse the topology of the associated data chain, prioritize the recovery of nodes with high impact factors and rebuild the dependency path in parallel.

[0042] Specific examples are as follows:

[0043] Scenario: Backup of multiple data sources in a financial transaction system.

[0044] S1 execution:

[0045] The call chain of "transaction records (high-frequency updates) → risk control logs (medium-frequency updates) → audit trails (low-frequency updates)" was detected.

[0046] Generate activity profile: Time sensitivity weight = [0.9, 0.6, 0.3], Influence factor = [0.8, 0.5, 0.2];

[0047] S2 execution:

[0048] Divide the time windows for differences (transaction record window = 5ms, others = 20ms);

[0049] When the 5ms window closes, perform an atomic snapshot on the transaction records and the strongly correlated risk control logs, generating the identifier "FIN_TX_5ms_A1_B2";

[0050] S3 execution:

[0051] The identifier was analyzed to identify transaction record (A1) as a high-impact node, which was then recovered first.

[0052] Parallel recovery of risk control logs (B2) and audit trails (via other identifiers).

[0053] S1 specifically includes:

[0054] S11, Update Frequency Quantization: For each data stream D i Create a sliding time window of length T, and count the timestamp t of each data update within the window. j And by adding a time decay factor, the dynamic update frequency value is calculated:

[0055] Among them, F i Represents data stream D i The dynamic update frequency value, n i Data stream D within the time window i The number of update events, t j λ represents the time elapsed between the j-th update event and the start of the time window. λ is the time decay factor (controlling the degree of decay of historical updates), with a value range of 0.01 to 0.1, adjusted according to the length of the time window, and a default value of 0.05. T represents the total length of the sliding time window.

[0056] S12, Dependency Resolution: Extract the call chain identifiers embedded in each data stream, construct a data dependency graph G = (V, E), and calculate the betweenness centrality C of each node v ∈ V. B (v), as the initial impact factor:

[0057] Where, σ st Let σ be the number of shortest paths from node s to node t. st (v) represents the number of shortest paths passing through node v, C B (v) is the betweenness centrality value of node v in the data dependency graph.

[0058] In a data dependency graph, G represents the data dependency graph, which is the structure of calls or dependencies between multiple data flows; V represents the set of nodes in the graph, where each node represents a data flow or data unit; and E represents the set of edges in the graph, where each edge represents the dependency relationship between one data flow and another data flow (in the direction of "depended on → dependent").

[0059] S13, Activity spectrum synthesis: First, the dynamically updated frequency value F is... i Normalized to time sensitivity weights:

[0060] α i It is a time-sensitive weight; then combined with the betweenness centrality C of each node in the dependency graph. B (v) and dependency depth d v (i.e., the shortest path length from the node to the source node), adjust its cross-datachain impact factor: β v It is the cross-datachain influence factor of node v, d v It represents the dependency depth, where γ∈(0,1) represents the dependency depth penalty factor (representing the proportion of cross-data chain influence factors retained for each deeper dependency chain; a value of γ=0.7 is recommended).

[0061] When the update frequency of a node exceeds the set threshold θ F At that time, the dynamic frequency coupling mechanism is triggered, and a promotion operation is performed on all its direct downstream nodes u∈succ(v): β u =β u +η·α v , where θ F η is the update frequency threshold, set as the mean of the update frequency within the sliding window plus one standard deviation. η is the frequency coupling gain factor, used to control the enhancement strength of the frequency-driven effect. sec(v) is the set of direct downstream dependent nodes of node v.

[0062] Ultimately, for all nodes v∈V, (α) v ,β v (α) forms an activity profile, which drives subsequent backup and recovery strategies. v ,β v ) represents an ordered pair consisting of two core attributes of node v in the activity graph, α v It is a time-sensitivity weight, reflecting the update activity of the data stream corresponding to node v. It is a normalized dynamic update frequency value, and its influence factor is β. v The structural importance of node v in cross-data chain dependencies is reflected by the result after correction based on betweenness centrality and dependency depth, (α) v ,β v) is defined as the activity feature vector of each node v in the activity graph, used to jointly evaluate its backup priority and recovery strategy weight.

[0063] S2 specifically includes:

[0064] S21, Dynamic Window Calculation: Based on the time sensitivity weights of data chains in the activity graph, dynamically determine the backup window length for each data chain. Among them, W i T represents the actual backup time window length of data chain i. base The baseline time window duration set for the system is 5 seconds, α i The time-sensitivity weight for data chain i is obtained from the normalized update frequency. This is a dynamic adjustment coefficient that reflects the sensitivity to activity level adjustments. It supports adaptive adjustment based on system load, with a value range of 1.0-3.0. It can be dynamically increased when the load is high, with a default value of 2.0.

[0065] S22, Atomic Snapshot Trigger: Within each time window, if an update event is detected in any node of the associated data chain, the window countdown is immediately reset. When the window closes, the following operations are performed:

[0066] A two-phase locking (2PL) protocol is adopted to acquire a shared lock on associated nodes when the data state is frozen;

[0067] Perform a consistent snapshot of the data state of all associated nodes and bind them to the same version number.

[0068] The "related data chain" refers to a set of nodes with direct or indirect dependencies constructed through a dependency graph. In other words, in the cross-dataflow dependency graph G=(V,E) constructed based on the call chain identifier, it is the set of all downstream or upstream nodes that can be traversed from a certain node through edge E. In short, the related data chain is a complete subgraph or subchain structure that forms a dependency path in the active graph. When a snapshot is triggered, all nodes on this chain are regarded as "related nodes" and need to be frozen and participate in atomic backup.

[0069] S23, Identifier Encoding:

[0070] A globally unique spatiotemporal consistency identifier is generated for each snapshot, with the following encoding structure:

[0071] SID = [t start ]-[t end ]-[H topo ]-[CRC]; The meanings of each field are as follows:

[0072] t startt is the start timestamp of the snapshot window. end H closes the timestamp for the snapshot window topo The topological hash value is the result of a hash function calculating the sequence of influence factors of each node in the current data chain. CRC is a 32-bit cyclic redundancy check code (CRC32) used for tamper-proof verification.

[0073] The method for generating topological hash values ​​is expressed as follows: in, It is the cross-datachain influence factor of the k-th node in the data chain, where n represents the number of nodes in the data chain, and SHA256(·) represents the standard secure hash algorithm used to generate irreversible topological fingerprints.

[0074] S3 specifically includes:

[0075] S31, Topology Reconstruction: Parsing the Topology H value H in the Spatiotemporal Consistency Identifier topo The system searches for matching terms in a pre-defined topological fingerprint database and reconstructs the dependency graph structure G=(V,E) in the original activity graph accordingly. It also adds the cross-datalink influence factor for each node at the recovery time. This represents the real-time impact factor of node v during recovery, which can be recalculated or obtained by looking up a table based on the current system state.

[0076] S32, Prioritization of Recovery: Based on all nodes v∈V in the graph structure G, generate the main recovery queue Q by arranging them in descending order of impact factor. main :

[0077] Sort indicates a sorting operation, arranging nodes in descending order of impact factor; descending indicates descending order.

[0078] Simultaneously, all sets of nodes with an in-degree of zero are identified, forming a parallel recovery sub-queue:

[0079] Q parallel ={v∈V|in-degree(v)=0}, where in-degree(v) represents the in-degree of node v, i.e., the number of its predecessor nodes.

[0080] S33, phased recovery execution, scheduling the recovery operation in three phases:

[0081] Phase i: Master Node Recovery and Preloading: From Q main Select all nodes from the queue that meet the following conditions: Trigger its state recovery and synchronously start the preloading process of its direct dependencies to accelerate subsequent unlocking operations, where θ is the high impact factor threshold, which takes a value between 0.7 and 0.9, and is set according to the criticality of the system;

[0082] Phase ii: Parallel node recovery: for Q parallel All nodes in the queue start multi-threaded concurrent recovery tasks and continuously monitor the recovery completion status of the main queue nodes in their dependent paths to dynamically unlock downstream dependencies.

[0083] Phase iii: Circular Dependency Handling: If a cycle is detected in the dependency graph... Implement the following loop-resolving strategy:

[0084] Calculate each edge e in the ring ij Edge influence factor corresponding to ∈C Take the edge corresponding to the minimum value as the cutting target: Remove the edge from the graph and generate a conflict marker:

[0085] ConflictTag=[CycleID]_Hash(e * )_Timestamp, where, Representing edge e ij Influence factor, the influence factor inherited from the source node, Hash(e * ) represents the unique hash fingerprint of the cut edge, Timestamp represents the current timestamp, and ConflictTag is the conflict marker record, used for log auditing and manual intervention.

[0086] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0087] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A backup and recovery management method based on multiple data sets, characterized in that: Includes the following steps: S1. Dynamic Data Activity Awareness: Real-time monitoring of the update frequency and dependencies of multiple data streams, generating an activity map including time-sensitivity weights and cross-data-chain influence factors; S1 specifically includes: The quantification of update frequency specifically includes creating a sliding time window for each data stream, counting the number of data updates within the sliding time window and adding a time decay factor to generate an update frequency value; The resolution of dependencies specifically includes extracting the call chain identifiers embedded in the data stream, constructing a dependency graph across the data stream, and calculating the betweenness centrality of each node in the dependency graph as an initial influence factor. The update frequency value is normalized to generate time sensitivity weights. At the same time, the cross-data chain influence factor is iteratively corrected based on the betweenness centrality and dependency depth in the dependency graph, and finally the activity graph is generated. S2. Spatiotemporal Consistency Backup: Dynamically divide time windows based on the activity profile, and perform atomic snapshot backups of the associated data chains when the time window closes, generating a spatiotemporal consistency identifier; specifically including: S21, Window dynamic calculation: Determine the length of the backup time window for each data link based on the time sensitivity weight value in the activity map; S22, Atomic Snapshot Trigger: When an update event is detected in any node of the associated data chain, the backup time window countdown of the chain is reset. When the backup time window is closed, the state of all associated nodes is frozen and a consistent snapshot is generated. Here, the associated data chain represents a set of nodes with direct or indirect dependencies constructed through a dependency graph. S23, Identifier Encoding: The snapshot timestamp, data chain topology hash value and check code are concatenated according to a preset format to generate a spatiotemporal consistency identifier, wherein the topology hash value comes from the cross-data chain influence factor distribution in the activity graph; S3. Topology-Driven Recovery: Based on spatiotemporal consistency identifiers, the topology of associated data chains is reverse-analyzed, prioritizing the recovery of nodes with high cross-link influence factors and reconstructing dependency paths in parallel; specifically including: S31, Topology Reconstruction: Parse the topology hash value in the spatiotemporal consistency identifier to restore the node dependencies and cross-data chain influence factors recorded in the live graph; S32, Recovery Priority Sorting: Nodes are sorted in descending order of cross-datalink impact factor to generate the main recovery queue, while nodes without predecessor dependencies in the topology are identified to form parallel recovery sub-queues; S33, phased recovery.

2. The backup and recovery management method based on multiple data sources according to claim 1, characterized in that, The modified cross-datalink impact factor is expressed as follows: ,in, This is the revised cross-datalink impact factor. For nodes Betweenness centrality value in dependency graphs This indicates a dependency depth penalty factor. This represents the dependency depth, which is the shortest path length from the node to the source node. This indicates a dependency on the depth penalty factor.

3. The backup and recovery management method based on multiple data sources according to claim 2, characterized in that, The generation of the activity graph specifically includes normalizing the update frequency value of the data corresponding to each node to obtain a time sensitivity weight at a unified scale, which characterizes the activity level of the data in the current time period. Combining the betweenness centrality of each node in the dependency graph with its dependency depth in the topology, a dependency depth penalty factor is applied to the betweenness centrality to obtain the initial cross-data chain influence factor of each node, suppressing the influence weight of deep indirect dependent nodes in the global topology. When the time sensitivity weight of a node exceeds a preset threshold, a dynamic frequency coupling mechanism is activated to adjust the gain of the cross-data chain influence factor of its directly downstream dependent nodes, strengthening the driving role of highly active nodes in the downstream nodes in its dependency path. The time sensitivity weight of each node is combined with the corrected cross-data chain influence factor to form an activity graph that supports the spatiotemporal consistency backup and topology-driven recovery strategy.

4. The backup and recovery management method based on multiple data sources according to claim 1, characterized in that, The backup time window length for each data chain is calculated as follows: ,in, Represents a data chain The actual backup time window length, The set base time window duration, For data link Time sensitivity weight, This is the dynamic adjustment coefficient.

5. The backup and recovery management method based on multiple data sources according to claim 1, characterized in that, The phased recovery process includes three phases, in which: Phase 1: Extract high cross-data chain impact factor nodes with values ​​above the preset threshold from the head of the main recovery queue and simultaneously start the preloading of all their dependencies; Phase 2: Initiate multi-threaded concurrent recovery for nodes in the parallel recovery sub-queue, while monitoring the dependency satisfaction status of nodes in the main recovery queue; Phase 3: When a circular dependency is detected, unlooping is forced based on the cross-data chain influence factor and conflict markers are recorded.

Images