Method for data synchronization in a disaster recovery backup system

Abstract

The application relates to the technical field of data synchronization, in particular to a data synchronization method in a disaster recovery backup system, which comprises the following steps: based on data blocks to be executed in a main data center, the data blocks are divided into a node set according to a preset byte length, then a directed edge set between nodes is defined according to the adjacent position relationship of the nodes in the original data blocks, and a data graph is obtained. The application divides the data blocks at the byte level, constructs a node set, defines a directed edge set in combination with the adjacent relationship of the nodes in the original data blocks, forms a data graph with a structured feature, can convert the linear storage form of the original data blocks into an information structure with a topological level, and realizes high compression and consistent expression of the relationship between nodes across structures by means of explicit mapping of the connection relationship by a topological adjacency matrix and generation of a global topological check value by one-way hash calculation.

Description

Technical Field

[0001] This invention relates to the field of data synchronization technology, and in particular to a method for data synchronization in a disaster recovery backup system. Background Technology

[0002] The field of data synchronization technology involves mechanisms and methods for maintaining data content consistency across multiple devices, systems, or locations, ensuring that the data status of each participating end can be updated to a consistent state in a timely and accurate manner after data is added, modified, or deleted.

[0003] Current technologies generally rely on comparing the data content itself for synchronization judgment. The process depends on data timestamps, version numbers, or change logs for multi-device synchronization. However, when the data structure remains largely unchanged but content details slightly shift, it often fails to accurately identify differences, leading to the risk of misjudgment. Furthermore, existing solutions use data blocks as the smallest unit of synchronization and lack internal structure modeling based on content relationships. This means that when performing cross-system reconciliation or backup data verification, only full-domain comparisons of the entire block's checksum are possible, making it difficult to identify which part of the data caused the difference, reducing synchronization efficiency and increasing resource overhead. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for data synchronization in a disaster recovery backup system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method for data synchronization in a disaster recovery backup system, comprising the following steps:

[0006] Based on the data blocks of the synchronous transactions to be executed in the main data center, the data blocks are divided into node sets according to a preset byte length, and then the set of directed edges between nodes is defined according to the adjacency relationship of the nodes in the original data blocks to obtain the data graph.

[0007] Based on the data graph, the connection relationships between all nodes are extracted and mapped into a two-dimensional array to obtain the topological adjacency matrix. Based on the topological adjacency matrix, a one-way hash operation is performed on all elements in the matrix to obtain the global topological verification value.

[0008] Based on the data graph, starting from the node with the smallest lexicographical order in the data graph, a deterministic node path is obtained. Based on the deterministic node path, the original data content corresponding to each node is extracted sequentially according to the node order of the path and chain hash operation is performed to generate a path sequence verification value.

[0009] Within the backup data center, based on the received data block replicas, a backup data map is regenerated and the backup global topology check value and backup path sequence check value are calculated. A redundancy operation check pair is obtained, and the received global topology check value and path sequence check value are compared with the backup global topology check value and backup path sequence check value in the redundancy operation check pair to obtain the synchronization consistency confirmation status.

[0010] Preferably, the steps for obtaining the data map are as follows:

[0011] Based on the data blocks of the synchronous transactions to be executed in the main data center, the data blocks are continuously truncated into several segments according to the set byte length parameter. The original byte sequence of each segment is extracted in turn and assigned a unique number to generate a set of byte sequence nodes with index identifiers.

[0012] Based on the set of byte sequence nodes with index identifiers, the connectivity value is calculated as follows:

[0013] Based on the byte sequence content of each node, a unique content fingerprint is generated using SHA-256 operation. A hash mapping table is constructed to map the fingerprint value to the corresponding set of node numbers. Two types of connection pairs are selected to form a candidate set: one type is adjacent node pairs with consecutive numbers, and the other type is non-consecutive node pairs with completely identical fingerprint values. The directed connectivity value is calculated only for each pair of nodes in the candidate set.

[0014] Based on the directed connectivity value, a directed edge structure is established, and the directed connectivity value is used as the edge weight. The node set and the edge structure are combined to generate a data graph.

[0015] Preferably, the step of obtaining the topological adjacency matrix is ​​as follows:

[0016] Based on the data graph, each directed edge is traversed from the directed edge structure of the data graph, and the index number of the starting node and the index number of the target node of each directed edge are extracted one by one to form a complete set of node connection pairs.

[0017] Based on the complete set of node connection relationships, the total number of index numbers of all nodes in the node set is counted. Using the node index number as the row and column coordinate identifier, a two-dimensional array is constructed with the number of rows and columns being the total number of node index numbers and the initial value of each element being zero.

[0018] Based on the two-dimensional array, each connection pair in the set of node connection pairs is mapped one by one. The row position of the two-dimensional array is determined by the starting node index number, and the column position of the two-dimensional array is determined by the target node index number. The element value at the corresponding position is marked as 1 to generate a topological adjacency matrix.

[0019] Preferably, the step of obtaining the global topology verification value is as follows:

[0020] Based on the topological adjacency matrix, the elements in the topological adjacency matrix are traversed row by row. The binary values ​​of each element in the matrix are extracted in order from top to bottom and from left to right. The extracted binary values ​​are then concatenated into a single continuous binary value sequence to generate the binary value sequence of matrix elements.

[0021] Based on the binary numerical sequence of the matrix elements, the binary numerical sequence of the matrix elements is input into a one-way hash function, and the hash mapping output of the binary numerical sequence of the matrix elements is calculated byte by byte to generate a hash digest value with a fixed byte length, thus obtaining a global topology verification value.

[0022] Preferably, the step of obtaining the deterministic node path is as follows:

[0023] Based on the data graph, all node indices in the data graph are traversed, the byte sequence content corresponding to each node is read in turn, and the character-by-character lexicographical comparison is performed according to the character encoding order. The node with the smallest lexicographical value among all nodes is selected as the traversal starting point, and the starting node index value is generated.

[0024] Based on the starting node index value, all paths reachable from that node are sequentially searched in the data graph. For each path, the difference between the connectivity degree value and the connecting node index of all edges is extracted in the order of the connecting edges, and the path importance value is calculated.

[0025] Based on the path importance values ​​of all paths, compare the path importance values ​​of all paths and select the unique path with the highest path importance value. Extract the order of node composition according to the connection order of node indexes in the path to generate a deterministic node path.

[0026] Preferably, the step of obtaining the path sequence verification value is as follows:

[0027] Based on the deterministic node path, the original byte sequence data content stored in each node of the deterministic node path is extracted node by node. The original byte sequence data content of each node is spliced ​​together according to the node order in the deterministic node path to form a continuous and complete original byte sequence data stream, thus generating a continuous original byte sequence data stream.

[0028] Based on the continuous raw byte sequence data stream, the data stream is divided into multiple continuous data segments of fixed byte length. A one-way hash operation is performed on each data segment in turn to obtain the segment hash value. Then, the hash value of the previous segment is concatenated with the current data segment to perform the next round of one-way hash operation. This process is repeated step by step to form a hash digest sequence and generate a path sequence verification value.

[0029] Preferably, the step of obtaining the redundant operation check pair is as follows:

[0030] Based on the data block copies received from the backup data center, a backup data map is regenerated and the backup global topology check value and backup path sequence check value are calculated.

[0031] Preferably, the step of obtaining the synchronization consistency confirmation status is as follows:

[0032] Based on the received global topology check value and the redundancy operation check pair, the byte code of the received global topology check value and the backup global topology check value in the redundancy operation check pair is extracted byte by byte and bit by bit, and compared bit by bit in order to determine the byte code difference value and count the total number of difference bits to generate a global topology difference statistical value.

[0033] Based on the received path sequence check value and the redundancy operation check pair, the byte encoding of the received path sequence check value and the backup path sequence check value in the redundancy operation check pair is extracted byte by byte and bit by bit. The byte encoding difference value is determined and the total number of difference bits is calculated to generate path sequence difference statistics.

[0034] Based on the global topology difference statistics and the path sequence difference statistics, it is determined whether both the global topology difference statistics and the path sequence difference statistics are equal to zero. If both the global topology difference statistics and the path sequence difference statistics are zero, the synchronization consistency status is confirmed as successful. If either the global topology difference statistics or the path sequence difference statistics are greater than zero, the synchronization consistency status is confirmed as failed, thus obtaining the synchronization consistency confirmation status.

[0035] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0036] This invention divides data blocks into byte-level segments and constructs a set of nodes. It then defines a set of directed edges based on the adjacency relationships of nodes within the original data blocks, forming a data graph with structured features. This transforms the linear storage form of the original data blocks into an information structure with topological hierarchy. By explicitly mapping connection relationships using a topological adjacency matrix and generating a global topological verification value through one-way hash calculation, it achieves high compression and consistent expression of relationships between cross-structure nodes. By extracting the lexicographically smallest node as the path starting point and determining the path by traversing the node with the highest number of nodes, and performing chained hashing operations to generate path sequence verification values, the path information possesses both content and structural determinism. The same structure is reconstructed on the backup end, and the verification value is repeatedly calculated. Bidirectional verification is achieved through redundant comparison, ensuring that consistency verification not only relies on the data itself but also incorporates structural mapping and access paths, improving the ability to capture fine-grained changes in data. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the steps of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0039] Please see Figure 1 This invention provides a technical solution, a method for data synchronization in a disaster recovery backup system, comprising the following steps:

[0040] Based on the data blocks of the synchronous transactions to be executed in the main data center, the data blocks are divided into node sets according to a preset byte length, and then the set of directed edges between nodes is defined according to the adjacency relationship of the nodes in the original data blocks to obtain the data graph.

[0041] Based on the data graph, the connection relationships between all nodes are extracted and mapped into a two-dimensional array to obtain the topological adjacency matrix. Based on the topological adjacency matrix, a one-way hash operation is performed on all elements in the matrix to obtain the global topological verification value.

[0042] Based on the data graph, the node with the smallest lexicographical order in the data graph is used as the starting point to obtain the deterministic node path. Based on the deterministic node path, the original data content corresponding to each node is extracted in sequence according to the node order of the path and chain hash operation is performed to generate the path sequence verification value.

[0043] Within the backup data center, based on the received data block replicas, a backup data map is regenerated, and the backup global topology check value and backup path sequence check value are calculated. A redundancy operation check pair is obtained, and the received global topology check value and path sequence check value are compared with the backup global topology check value and backup path sequence check value in the redundancy operation check pair to obtain the synchronization consistency confirmation status.

[0044] The steps for obtaining the data map are as follows:

[0045] Based on the data blocks of the synchronous transactions to be executed in the main data center, the data blocks are continuously truncated into several segments according to the set byte length parameter. The original byte sequence of each segment is extracted in turn and assigned a unique number to generate a set of byte sequence nodes with index identifiers.

[0046] The connectivity value is calculated based on the set of byte sequence nodes with index identifiers. The calculation process is as follows:

[0047] A unique content fingerprint is generated based on the byte sequence content of each node using SHA-256 computation. By constructing a hash mapping table to map fingerprint values ​​to the corresponding set of node numbers, two types of connection pairs are selected to form a candidate set. One type consists of consecutively numbered pairs of adjacent nodes. Another type consists of non-contiguous node pairs with identical fingerprint values. Only for each pair of nodes in the candidate set The directed connectivity value is calculated using the following formula:

[0048] ;

[0049] in, Indicates the first The node and the first The directed connectivity value between nodes. Indicates the first Shannon entropy of each node, Indicates the first Shannon entropy of each node, Indicates the first The node and the first The joint Shannon entropy of the content obtained by concatenating the byte sequences of each node. Indicates the first The length of each node in bytes. Indicates the first The length of each node in bytes;

[0050] Based on the directed connectivity value, a directed edge structure is established and the directed connectivity value is used as the edge weight. The node set and the edge structure are combined to generate a data graph.

[0051] Specifically, based on the data blocks in the main data center to be synchronized, the byte length parameter is first determined according to the content type of the data block and the expected synchronization network bandwidth. For example, for structured database files, this parameter is set to 4096 bytes, and for unstructured log files, it is set to 1024 bytes. Taking a database file to be synchronized with a size of 10MB (10,485,760 bytes) as an example, it is processed using a length parameter of 4096 bytes. Starting from the beginning of the file, 4096 bytes of data are read sequentially, treating each byte as an independent segment. This process will continue until the end of the file, generating a total of [number missing] bytes. The system truncates each of the 2560 segments by assigning a unique identifier to each segment. If the last segment is less than 4096 bytes, it is truncated to its remaining length. The system then assigns each segment a unique identifier to a consecutive integer from 1 to 2560. Each segment's original byte sequence is bound to its unique identifier, forming a structured object containing an index and content. For example, node number 1 corresponds to the first 4096 bytes of the file, node number 2 corresponds to bytes 4097 to 8192, and so on. Finally, all these structured objects carrying unique identifiers and original byte sequences are combined to generate a set of byte sequence nodes with index identifiers.

[0052] formula: The advantage of this formula lies in its combination of two dimensions—"information complexity" and "physical size" within a data block—to quantify the connection strength between nodes. This is achieved by introducing Shannon entropy. and Entropy measures the randomness and uncertainty of the information contained in each data node; a higher entropy value indicates more disordered data and a greater amount of information. The combined Shannon entropy... The overall information complexity after merging the two nodes was evaluated. This ratio effectively identifies the content similarity and redundancy between nodes. When two nodes have highly similar content or strong correlation, the increase in joint entropy will be less than the sum of individual entropies, leading to an increase in this ratio. Furthermore, by introducing a logarithmic node length term... This allows the connectivity value to increase moderately with the increase of the total node length, but at a slower rate. This takes into account the impact of the amount of data and avoids the problem of excessive amplification of the connectivity value due to excessively large nodes. As a result, it can more evenly and accurately depict the inherent logical relationship between nodes in the data graph.

[0053] Indicates the first The Shannon entropy of a node is used to measure the information content and uncertainty of the byte sequence of that node. It is obtained by first statistically analyzing the nodes... Count the number of occurrences of each byte value (0-255) in all byte sequences, and then calculate the probability of occurrence of each byte value. Finally, substitute into the Shannon entropy calculation formula Perform calculations. For example, for a 4-byte node... Its content is [0xAA, 0xBB, 0xAA, 0xCC], with byte 0xAA appearing twice, and 0xBB and 0xCC each appearing once, for a total of 4 bytes. Therefore... , , .therefore, .

[0054] Indicates the first The Shannon entropy of each node is calculated in the same way as... Completely the same, except it applies to the first... The byte sequence of each node. For example, for another 4-byte node. Its content is [0xAA, 0xBB, 0xDD, 0xEE], with bytes 0xAA, 0xBB, 0xDD, and 0xEE each appearing once, for a total of 4 bytes. Therefore... .therefore, .

[0055] Indicates the first The node and the first The joint Shannon entropy is the result of concatenating the byte sequences of two nodes. It is obtained by first merging the byte sequences of the two nodes sequentially into a longer sequence, and then using this new sequence to calculate... The same method is used to calculate its overall Shannon entropy. Continuing with the example above, the node... and The concatenated sequence is an 8-byte sequence [0xAA, 0xBB, 0xAA, 0xCC, 0xAA, 0xBB, 0xDD, 0xEE]. The number of occurrences of each byte is: 0xAA 3 times, 0xBB 2 times, and 0xCC, 0xDD, and 0xEE 1 time each. The probabilities are calculated as follows: , , , , .therefore, .

[0056] Indicates the first The byte length of each node is determined by directly counting the nodes. The value is obtained by determining the total number of bytes contained within the data block. During synchronization, since data blocks are segmented according to a preset byte length, this value is usually fixed, unless it is the last node in the data block. For example, as mentioned earlier, a node... The byte sequence is [0xAA, 0xBB, 0xAA, 0xCC], and its byte length is... It is 4 bytes.

[0057] Indicates the first The byte length of each node, and how it is obtained. Completely identical, i.e., statistical nodes The total number of bytes contained. For example, according to the previous text, the node The byte sequence is [0xAA, 0xBB, 0xDD, 0xEE], and its byte length is [0xAA, 0xBB, 0xDD, 0xEE]. It is 4 bytes.

[0058] Calculation process:

[0059] With nodes and nodes For example, the obtained parameter values ​​are substituted into the formula for calculation to obtain the directed connectivity value between the two nodes. .

[0060] The known parameters are as follows:

[0061] ;

[0062] ;

[0063] ;

[0064] ;

[0065] ;

[0066] The calculation process is as follows:

[0067] ;

[0068] ;

[0069] ;

[0070] ;

[0071] This result indicates that the node With nodes The directed connectivity between the two data nodes is 4.869. This value quantitatively describes the strength of the structural association between the two data nodes. The higher the value, the closer their association in terms of content and structure.

[0072] Based on the directed connectivity values, the system first sets an edge weight threshold for filtering valid connections. This threshold is set with reference to the statistical distribution characteristics of the directed connectivity values ​​calculated for all candidate connection pairs. Specifically, the calculation method involves calculating the mean and standard deviation of all connectivity values, and then setting the threshold to the mean plus 1.2 times the standard deviation. For example, if the mean of all connectivity values ​​is 3.5 and the standard deviation is 0.8, then the edge weight threshold is set to... Next, the system iterates through all the calculated directed connectivity values. Only when A node is confirmed only when the value is greater than 4.46. With nodes There exists a valid directed edge, and the directed connectivity value is set to... The weight is assigned to this edge. For connectivity values ​​less than or equal to 4.46, they are ignored and no corresponding edge is created. This method filters out a large number of weak or accidentally formed connections. Then, all the filtered node pairs are... and their corresponding edge weights This is combined into a set of directed edge structures. Finally, this set of directed edge structures is integrated with the set of byte sequence nodes with indexed identifiers generated in the previous step. Specifically, this is implemented by constructing an adjacency list data structure indexed by node number, where each node entry lists all directed edges originating from that node, their target nodes, and corresponding edge weights, thus ultimately generating a complete, weighted, and directed data graph.

[0073] The steps to obtain the topological adjacency matrix are as follows:

[0074] Based on the data graph, each directed edge is traversed from the directed edge structure of the data graph, and the index number of the starting node and the index number of the target node of each directed edge are extracted one by one to form a complete set of node connection pairs.

[0075] Based on the complete set of node connections, the total number of index numbers of all nodes in the set is counted. Using the node index number as the row and column coordinate identifier, a two-dimensional array is constructed with the number of rows and columns equal to the total number of node index numbers and the initial value of each element being zero.

[0076] Based on a two-dimensional array, each connection pair in the set of node connection pairs is mapped one by one. The row position of the two-dimensional array is determined by the index number of the starting node, and the column position of the two-dimensional array is determined by the index number of the target node. The element value at the corresponding position is marked as 1, and a topological adjacency matrix is ​​generated.

[0077] Specifically, based on the data graph, a depth-first traversal algorithm is used to systematically access each node and all its outgoing edges in the data graph. Specifically, from the adjacency list representation of the data graph, nodes are processed sequentially starting from index 1. For the currently processed node... The program will read all outgoing edges recorded in its adjacency list, and each outgoing edge contains the index number of the target node. and the corresponding edge weight During this process, only the index number of the starting node is extracted. and the index number of the target node Ignoring edge weight information, each extracted pair As an independent node connection pair, it is added to a temporary set. For example, if the data graph contains nodes 1 to 5, and there are valid directed edges from node 1 to node 3, from node 1 to node 5, and from node 2 to node 4, then the traversal process will first process node 1 and extract the connection pairs. and Next, process node 2 and extract the connection pairs. Then, the subsequent nodes are processed until all outgoing edges of all nodes have been visited. This traversal process ensures that every directed edge in the data graph is visited exactly once. Finally, all the connection pairs collected in the temporary set are summarized to form a complete set of node connection pairs.

[0078] Based on the complete set of node connection relationships, the system first needs to determine the total number of nodes in the data graph. This is done by retrieving the set of indexed byte sequence nodes generated in the initial step, directly obtaining the total number of nodes contained within it, denoted as . Alternatively, by traversing the complete set of node connections, the maximum value among all occurrences of the starting node index and the target node index can be used as the [value]. For example, if the byte sequence node set contains 2560 nodes, then The value is 2560. Subsequently, the system dynamically creates a two-dimensional integer array in memory, the dimension of which is set to 2560. That is, a square matrix containing 2560 rows and 2560 columns, where the rows and columns are numbered from 0 to 1. Integers are used for indexing, and these indices are related to the node's index number (usually from 1 to 1). Establish a direct mapping relationship, that is, the first element of the array... row and number The columns correspond to index numbers respectively. and When creating this two-dimensional array, all elements within the node are initialized to zero. This operation constructs an empty matrix structure that corresponds perfectly to the size of the data graph nodes, providing the foundation for subsequent mapping of node connections. A two-dimensional array is created with the number of rows and columns equal to the total number of node index numbers and the initial value of each element being zero.

[0079] Based on a two-dimensional array, process the node connection pairs one by one for each connection pair in the set. ,in This is the index number of the starting node. The index number of the target node is used. For each read connection pair, the program determines its specific position in the two-dimensional array according to a pre-established mapping rule, i.e., the array index equals the node index number minus one. Specifically, the starting node index number is used. Calculate the corresponding row position as Use the target node index number Calculate the corresponding column position as For example, for a connection pair The corresponding position in the two-dimensional array is row 2, column 7 (index starts from 0), and the program then accesses this position. The elements in the node connection pair are mapped and their values ​​are changed from the initial 0 to 1. This modification operation indicates that there is a directed connection from node 3 to node 8. This process continues until all elements in the node connection pair set have been processed and all corresponding two-dimensional array positions are marked as 1. For node connections that do not appear in the connection pair, the element values ​​at the corresponding positions in the array will remain at the initial 0. Finally, this two-dimensional array after all connection mappings and marking modifications generates the topological adjacency matrix.

[0080] The steps to obtain the global topology check value are as follows:

[0081] Based on the topological adjacency matrix, the elements in the topological adjacency matrix are traversed row by row. The binary value of each element in the matrix is ​​extracted in order from top to bottom and from left to right. The extracted binary values ​​are then concatenated into a single continuous binary value sequence to generate the binary value sequence of matrix elements.

[0082] Based on the binary numerical sequence of matrix elements, the binary numerical sequence of matrix elements is input into a one-way hash function. The hash mapping output of the binary numerical sequence of matrix elements is calculated byte by byte to generate a hash digest value with a fixed byte length, and thus obtains the global topology verification value.

[0083] Specifically, based on the topological adjacency matrix, a nested loop structure is used. First, an empty character buffer is initialized to sequentially store the extracted binary values. The outer loop starts from the first row (row index 0) of the topological adjacency matrix and iterates downwards row by row until the last row. The inner loop starts from the first column (column index 0) within the current row and iterates to the right column by column until the last column. For each matrix element position determined by its row and column indices, the system reads the value stored at that position, which is always 0 or 1, and appends this numeric character directly to the end of the character buffer. For example, for a 3x3 topological adjacency matrix... The matrix [[0, 1, 1], [1, 0, 0], [0, 1, 0]] is traversed starting with element 0 at position [0][0] and appended to the buffer. Then, element 1 at position [0][1] is processed, and the buffer content becomes "01". Next, element 1 at position [0][2] is processed, and the content becomes "011". After completing the first row, the traversal moves to the second row and appends elements 1, 0, and 0 in sequence, updating the buffer content to "011100". Finally, the third row is processed, and 0, 1, and 0 are appended. When the last element in the matrix (i.e., 0 at position [2][2]) is extracted and concatenated, a binary numerical sequence of matrix elements is generated.

[0084] Based on the binary numerical sequence of matrix elements, the system first preprocesses this long string composed of '0' and '1' characters, converting it into a byte array. Specifically, starting from the beginning of the sequence, it divides the string into groups of 8 consecutive characters and parses each group of 8-bit binary strings into a corresponding 8-bit byte value. For example, the first 16 bits of the binary sequence "0111000101101100" will be converted into two bytes, namely 0x71 and 0x6C. If the total length of the sequence is not a multiple of 8, several '0's are added to the end of the sequence. The total length is made divisible by 8, and then converted. This processed byte array is then used as input to a specific one-way hash function, such as SHA-256. This function initializes its internal 256-bit hash value register and divides the input data into blocks of 512 bits (64 bytes). Each data block undergoes 64 rounds of compression transformation, and the final 256-bit hash value register content is output. This result is formatted as a 64-character hexadecimal string, generating a hash digest value with a fixed byte length, thus obtaining the global topology check value.

[0085] The steps to obtain the deterministic node path are as follows:

[0086] Based on the data graph, traverse all node indices in the data graph, read the byte sequence content corresponding to each node in turn, and perform character-by-character lexicographical comparison according to the character encoding order. Select the node with the smallest lexicographical value among all nodes as the traversal starting point and generate the starting node index value.

[0087] Based on the starting node index, all paths reachable from that node are sequentially searched in the data graph. For each path, the connectivity value of all edges is extracted in the order of the connecting edges, and the difference between the connectivity value and the connecting node index is calculated. The path importance value is calculated using the following formula:

[0088] ;

[0089] in, This represents the path importance value. This indicates the number of edges contained in the path. Indicates from the first The node points to the first The connectivity value of each node. This represents the absolute value of the difference between the starting and ending node indices of an edge;

[0090] Based on the path importance values ​​of all paths, compare the path importance values ​​of all paths and select the unique path with the highest path importance value. Extract the order of node composition according to the connection order of node indexes in the path to generate a deterministic node path.

[0091] Specifically, based on the data graph, a temporary variable is initialized to store the index of the currently discovered lexicographically smallest node and its corresponding byte sequence content. The content of the first node in the data graph (index 1) is assigned as the initial value to this variable. Subsequently, the program iterates from the second node in the data graph to the last node in order of index number. For each currently traversed node, the program reads the complete original byte sequence from its storage structure and compares it lexicographically with the smallest byte sequence stored in the temporary variable. This comparison process starts from the first byte of the two sequences. If the byte values ​​are different, the sequence corresponding to the smaller byte value is the lexicographically smaller sequence. If the byte values ​​are the same, the comparison continues to the next byte until the end of the sequence or a difference is found. For example, if the current smallest sequence is [0x0A, 0x01, 0xFF], and the node sequence being compared is [0x0A, 0x01, 0xFE], then when comparing the third byte, since 0xFE is less than 0xFF, the new node's sequence is lexicographically smaller. At this point, the program updates the temporary variable, replacing the node index in it with the index of the current node, and replacing its byte sequence content with the sequence of the current node. After this iterative comparison process covers all nodes in the data graph, the node index finally stored in the temporary variable is the node with the smallest global lexicographical order, generating the starting node index value.

[0092] formula: The advantage of this formula lies in its ability to comprehensively evaluate both the "internal connectivity" and "global traversal" of a path through a multiplicative structure, thereby identifying the most structurally representative core path in the data. (Left side) The average connectivity value of all nodes along the path is calculated, which tends to select paths composed of highly interconnected nodes, reflecting the local quality of the path. (Right side) This quantifies the total span of the path across node indices. A large span value means the path connects nodes that are physically far apart in the original data block. This usually indicates an important, non-local structural repetition or association, expressed as a natural logarithm. The function processes the summation of spans, which preserves the influence of the span size and prevents the problem of excessive weighting of a single extremely large span edge. Multiplying these two parts ensures that the final selected path is not only internally stable, but also has macroscopic importance in the entire data graph.

[0093] This represents the number of edges contained in the path, an integer obtained by direct counting. When performing path searching on a data graph, each movement from one node to the next adjacent node is considered as traversing an edge. For a given path, such as starting from the starting node and passing through a series of intermediate nodes to reach the destination, the number of edges it contains is the path length minus one. For example, if a searched path is node A->node B->node C->node D, this path traverses three edges in sequence. , and ,therefore The value is 3.

[0094] Indicates from the first The node points to the first The connectivity value of each node is determined during the data graph construction phase by analyzing nodes. and nodes The content complexity and length are calculated, and the specific calculation method refers to the directed connectivity value formula in the previous steps. This parameter is directly extracted from the edge attributes of the generated data graph. For example, when analyzing a path node 3->node 8->node 15, it is necessary to query the edges from the data graph separately. and edge The weights are used to obtain specific connectivity values. Based on the calculation results of the previous steps, a series of such values ​​can be obtained, such as... , .

[0095] This represents the absolute value of the difference between the start and end node indices of an edge. This parameter reflects the physical span of the edge within the original data block sequence. Only one edge needs to be retrieved. Starting node index and endpoint node index Then calculate the absolute value of the difference between the two. For example, for an edge... The absolute value of its index difference is For the edge The absolute value of its index difference is .

[0096] Calculation process:

[0097] Taking a reachable path path1 starting from the starting node as an example, the path is node 3-> node 8-> node 15.

[0098] Based on the path definition, the following parameters can be extracted:

[0099] The path contains the following edges: and Therefore, the number of edges .

[0100] Extract the connectivity value of each edge from the data graph to obtain... and .

[0101] Calculate the absolute value of the node index difference for each edge to obtain... and .

[0102] Substitute the above parameters into the path importance value formula for calculation:

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] ;

[0108] ;

[0109] The results show that the path importance value of the path node 3-> node 8-> node 15 is 14.4917. This value is a quantitative representation of the importance of the path in the data graph. The system will calculate the corresponding path importance value for all reachable paths starting from the same starting point, and identify the most important path by comparing these values. The higher the value, the greater the structural significance of the path, and the greater the possibility of it being selected as the final deterministic node path.

[0110] Based on the path importance values ​​of all paths, the system stores all calculated paths and their corresponding importance values ​​in a list, for example, [(path1, 14.4917), (path2, 9.8731), (path3, 15.0122)]. Next, the system initializes a variable to track the current maximum importance value and sets it as the importance value of the first path in the list, simultaneously recording this path as the current best path. Then, the system iterates from the second element of the list, comparing the importance value of each subsequent path with the currently recorded maximum value. If the current path's value is greater than the recorded maximum value, the maximum value is updated, and the best path is replaced with the current path. For example, after comparing path2, since 9.8731 is less than 14.4917, no update is performed; after comparing path3, since 15.0122 is greater than 14... 0.4917, the system updates the maximum value to 15.0122 and the optimal path to path3. To ensure the uniqueness of the filtering results, if two or more paths have the same maximum importance value, a preset deterministic tie-breaking rule is activated. This rule stipulates that in this case, the path with the smaller lexicographical order of the node index sequence will be selected. For example, if paths 3-8-15 and 3-9-12 have the same score, the former will be selected because the sequence (3, 8, 15) is smaller in lexicographical order than (3, 9, 12). After traversing all paths, the optimal path is finally determined, which is the unique path with the largest path importance value. Finally, the system extracts the index number of each node from this unique path according to the order in which the nodes are connected, and combines these numbers into an ordered list to generate a deterministic node path.

[0111] The steps for obtaining the path sequence check value are as follows:

[0112] Based on the deterministic node path, the original byte sequence data content stored in each node of the deterministic node path is extracted node by node. The original byte sequence data content of each node is spliced ​​together according to the node order in the deterministic node path to form a continuous and complete original byte sequence data stream, thus generating a continuous original byte sequence data stream.

[0113] Based on a continuous raw byte sequence data stream, the data stream is divided into multiple continuous data segments of fixed byte length. A one-way hash operation is performed on each data segment in turn to obtain the segment hash value. Then, the hash value of the previous segment is concatenated with the current data segment to perform the next round of one-way hash operation. This process is repeated step by step to form a hash digest sequence and generate a path sequence verification value.

[0114] Specifically, based on a deterministic node path, which is a list containing ordered node indices, such as [3, 8, 15], the system first initializes an empty byte buffer. Then, the program iterates according to the node index order recorded in the deterministic node path. For the first index 3 in the list, the system accesses the node storage structure of the data graph, directly locates the node with index 3, and reads its corresponding original byte sequence data content completely. For example, 4096 bytes of data are read, and the program directly appends this part of the data to the end of the byte buffer. Next, the second index 8 in the list is processed. Similarly, the system locates the node with index 8, reads all 4096 bytes of its original data, and then concatenates it to the existing content of the byte buffer. At this time, the buffer contains 8192 bytes of data. This process continues until the data of the last node (index 15) in the deterministic node path is also read and concatenated. Finally, the byte buffer will contain a byte stream that is seamlessly concatenated in order from the original data of all nodes on the path, generating a continuous original byte sequence data stream.

[0115] Based on a continuous raw byte sequence data stream, the system first determines a fixed byte length for segmentation, set to 256 bytes. This length is an empirical value that strikes a balance between computational efficiency and hash sensitivity. Then, the system initializes a hash value variable with a fixed, predefined 256-bit seed value, such as an all-zero hash value. Next, starting from the beginning of the continuous raw byte sequence data stream, the system processes the data in 256-byte blocks. For the first 256-byte data segment, the system uses the SHA-256 one-way hash function to calculate the first segment hash value, and then proceeds to the recursive calculation phase. For the second 256-byte data segment... The system first concatenates the first segment hash value (32 bytes) obtained in the previous step with the current second data segment (256 bytes) to form a 288-byte input data block. Then, this concatenated data block is fed into the SHA-256 function for calculation to obtain a new hash value. This new hash value will be used as the "previous segment hash value" for the next round of calculation. This process is repeated continuously. In each round, the hash output of the previous round is combined with the current data segment to perform a new round of hash calculation, forming a hash chain. This continues until all segments of the continuous original byte sequence data stream have been processed. The output result of the last hash operation is the final hash digest sequence, which generates the path sequence verification value.

[0116] The steps for obtaining the redundancy check pair are as follows:

[0117] Based on the data block copies received from the backup data center, a backup data map is regenerated and the backup global topology check value and backup path sequence check value are calculated.

[0118] Specifically, based on the data block replicas received from the backup data center, the system completely replicates the entire computation process on the primary data center side at the backup end. First, strictly adhering to the byte length parameters used by the primary data center, the data block replica is continuously truncated into several segments and assigned unique identifiers from 1 to N, generating a set of byte sequence nodes with indexed identifiers. Next, for this newly generated set of nodes, the entire process of content fingerprint calculation, candidate link pair filtering, and directed connectivity degree calculation is performed, using the same edge weight thresholds as the primary data center to filter effective connections. This constructs a backup data graph that is structurally equivalent to the primary data center's data graph. Subsequently, based on this backup data graph, through... By traversing the edge set, constructing an empty two-dimensional array, and mapping the connection relationships, a topological adjacency matrix for the backup end is generated. This matrix is ​​then serialized and input into the same SHA-256 hash function to calculate the backup global topological verification value. Simultaneously, based on the backup data graph, the system finds the traversal starting point through lexicographical comparison of all nodes, uses algorithms such as depth-first search to find all reachable paths, and evaluates each path using the same path importance value calculation formula as the master end. The unique path with the highest importance value is selected as the deterministic node path for the backup end. Finally, the contents of the backup deterministic node path are concatenated and chained hashed to generate the backup path sequence verification value.

[0119] The steps to obtain the synchronization consistency confirmation status are as follows:

[0120] Based on the received global topology check value and the redundancy operation check pair, the byte code of the backup global topology check value in the received global topology check value and the redundancy operation check pair is extracted byte by byte and bit by bit. The byte code is compared bit by bit in order to determine the difference value of the byte code and to calculate the total number of difference bits, thereby generating a global topology difference statistical value.

[0121] Based on the received path sequence check value and the redundancy operation check pair, the byte encoding of the backup path sequence check value in the received path sequence check value and the redundancy operation check pair is extracted byte by byte and bit by bit. The byte encoding difference value is determined and the total number of difference bits is calculated to generate the path sequence difference statistics.

[0122] Based on the global topology difference statistics and the path sequence difference statistics, it is determined whether both the global topology difference statistics and the path sequence difference statistics are equal to zero. If both the global topology difference statistics and the path sequence difference statistics are zero, the synchronization consistency status is confirmed as successful. If either the global topology difference statistics or the path sequence difference statistics are greater than zero, the synchronization consistency status is confirmed as failed, and the synchronization consistency confirmation status is obtained.

[0123] Specifically, based on the received global topology checksum and the redundancy checksum pair, a backup global topology checksum is extracted from the redundancy checksum pair. Both checksums are fixed-length byte sequences, such as 32-byte SHA-256 hash values. The program initializes a difference counter to zero, and then processes each byte sequentially, starting from the first byte of the two checksum byte sequences. For each pair of bytes, the program performs a bitwise XOR operation to obtain a result byte. In this result byte, a bit with a value of 1 indicates that the original two bytes differ at that bit, and a bit with a value of 0 indicates that they are the same. For example, if the received byte is 0b10110010 and the backup byte is 0b10100011, the XOR result is 0b00010001. Then, the program counts the number of bits with a value of 1 in this XOR result byte. This can be achieved by looking up a pre-compute table or by circular shift and bitwise AND operations. The counted difference bits are then added to the total difference counter. This process is applied sequentially to all 32 bytes in the two check value sequences. Finally, the value recorded in the difference counter is the total number of difference bits between the two global topology check values, generating a global topology difference statistics value.

[0124] Based on the received path sequence checksum and the redundancy checksum pair, the backup path sequence checksum is extracted from the redundancy checksum pair and placed alongside the path sequence checksum received from the main data center. The program also initializes a separate difference counter to zero and compares these two 32-byte hash values ​​byte-by-byte and bit-by-bit. For each pair of bytes at corresponding positions, a bitwise XOR operation is used to identify the differing bits. For example, if the received k-th byte is 0xAF (0b10101111), and the backup k-th byte... The first byte is 0xAE (0b10101110), and their XOR result is 0x01 (0b00000001), indicating that there is a difference in the least significant bit. The program adds this difference count (1 in this example) to the difference counter of the path sequence. This byte comparison and difference accumulation process will cover all 256 bits of the check values ​​of the two path sequences until the last byte comparison is completed. The final sum of the difference counters accurately quantifies the degree of inconsistency between the check values ​​of the two path sequences and generates the path sequence difference statistics.

[0125] Based on the global topology difference statistics and path sequence difference statistics, the system checks whether the global topology difference statistics are strictly equal to zero and whether the path sequence difference statistics are also strictly equal to zero. Only when both conditions are met, i.e., both statistics are zero, will the system determine that the data is completely consistent with the main data center at both the structural level and the core content sequence level. At this time, the system sets the internal synchronization consistency status flag to "synchronization successful" and records this result in the synchronization log. If at least one of the above two conditions is not met, i.e., the global topology difference statistics are greater than zero, or the path sequence difference statistics are greater than zero, or both are greater than zero, this indicates that at least one deviation has occurred during data transmission or backup reconstruction. Whether the structural relationship of the data blocks has changed or the key content sequence has mismatched, the system will set the synchronization consistency status flag to "synchronization failed" and trigger subsequent error handling or retransmission mechanisms to obtain a synchronization consistency confirmation status.

[0126] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for data synchronization in a disaster backup system, characterized in that, The method comprises the following steps: Based on the data block to be executed in the main data center, the data block is divided into a node set according to a preset byte length, and a directed edge set between nodes is defined according to the adjacent position relationship of the nodes in the original data block, to obtain a data graph; Based on the data graph, the connection relationship between all nodes is extracted and mapped into a two-dimensional array to obtain a topology adjacency matrix, and one-way hash operation is performed on all elements in the matrix based on the topology adjacency matrix to obtain a global topology check value; Based on the data graph, the smallest node in the dictionary order of the data graph is taken as a starting point to obtain a deterministic node path, and the original data content corresponding to each node is extracted in the order of the nodes of the path and subjected to chain hash operation based on the deterministic node path to generate a path sequence check value; In the backup data center, based on the received data block copy, a backup data graph is regenerated and a backup global topology check value and a backup path sequence check value are calculated to obtain a redundant operation check pair, and the received global topology check value and path sequence check value are compared with the backup global topology check value and backup path sequence check value in the redundant operation check pair to obtain a synchronization consistency confirmation state.

2. The method for data synchronization in a disaster backup system according to claim 1, wherein, The data graph obtaining step is: Based on the data block to be executed in the main data center, the data block is continuously truncated into several paragraphs according to a set byte length parameter, the original byte sequence of each truncated content is extracted in turn and is given a unique number identification to generate a byte sequence node set with index identification; According to the byte sequence node set with index identification, the connection degree value is calculated, and the calculation process is as follows: Based on the byte sequence content of each node, a unique content fingerprint is generated by using SHA-256 operation, the fingerprint value is mapped to the corresponding node number set by constructing a hash mapping table, and two types of connection pairs are selected to form a candidate set: one type is a pair of adjacent nodes with consecutive numbers, and the other type is a pair of non-consecutive nodes with the same fingerprint value, and only the directed connection degree value of each node pair in the candidate set is calculated; According to the directed connection degree value, a directed edge structure is established and the directed connection degree value is taken as the edge weight, the node set and the edge structure are combined to generate a data graph.

3. The method for data synchronization in a disaster backup system according to claim 1, wherein, The topology adjacency matrix obtaining step is: Based on the data graph, each directed edge in the directed edge structure of the data graph is traversed, the index number of the starting node and the index number of the target node of each directed edge are extracted one by one to form a complete node connection relationship pair set; Based on the complete node connection relationship pair set, the total amount of index numbers of all nodes in the node set is counted, and a two-dimensional array with the total amount of node index numbers as the row and column coordinates and with zero as the initial value of the elements is established; Based on the two-dimensional array, each connection relationship pair in the complete node connection relationship pair set is mapped one by one, the row position of the two-dimensional array is determined according to the index number of the starting node, the column position of the two-dimensional array is determined according to the index number of the target node, and the element value of the corresponding position is marked as 1 to generate a topology adjacency matrix.

4. The method for data synchronization in a disaster backup system according to claim 1, wherein, The global topology check value obtaining step is: Based on the topology adjacency matrix, elements in the topology adjacency matrix are traversed row by row, binary values of each element in the matrix are extracted in sequence from top to bottom and from left to right, and the binary values are spliced into a single continuous binary value sequence to generate a matrix element binary value sequence; Based on the matrix element binary value sequence, the matrix element binary value sequence is input into a one-way hash function, and the hash mapping output of the matrix element binary value sequence is calculated byte by byte to generate a hash digest value with a fixed byte length, and a global topology check value is obtained.

5. The method for data synchronization in a disaster backup system according to claim 1, wherein, The acquisition step of the deterministic node path is: Based on the data graph, all node indexes in the data graph are traversed, the byte sequence content corresponding to each node is read in sequence, and a character-by-character lexicographic comparison is performed according to the character encoding order, the node with the smallest lexicographic arrangement value among all nodes is selected as the starting point of traversal, and a starting node index value is generated; According to the starting node index value, all paths that can be reached from the node are found in the data graph in sequence, and the connection degree value and the connection node index difference value of all edges are extracted according to the order of the connected edges, and the path importance value is calculated; According to the path importance values of all paths, the path importance values of all paths are compared and the only path with the largest path importance value is filtered, the node composition order is extracted according to the connection order of the node indexes in the path, and a deterministic node path is generated.

6. The method for data synchronization in a disaster backup system according to claim 1, wherein, The acquisition step of the path sequence check value is: Based on the deterministic node path, the original byte sequence data content stored in each node in the deterministic node path is extracted node by node, and the original byte sequence data content of each node is spliced to form a continuous and complete original byte sequence data stream according to the node order in the deterministic node path, and a continuous original byte sequence data stream is generated; Based on the continuous original byte sequence data stream, the data stream is divided into multiple continuous data segments with a fixed byte length, and one-way hash operation is performed on each data segment in sequence to obtain a segment hash value, and the next round of one-way hash operation is performed after splicing the previous segment hash value and the current data segment together, and the hash digest sequence is recursively formed to generate a path sequence check value.

7. The method for data synchronization in a disaster backup system according to claim 1, wherein, The acquisition step of the redundant operation check pair is: Based on the data block copy received by the backup data center, a backup data graph is regenerated and backup global topology check values and backup path sequence check values are calculated.

8. The method for data synchronization in a disaster backup system according to claim 1, wherein, The acquisition step of the synchronization consistency confirmation state is: Based on the received global topology check value and the redundant operation check pair, the byte encoding of the received global topology check value and the backup global topology check value in the redundant operation check pair are extracted byte by byte, and the byte encoding difference value is determined and the total number of difference bits is counted, and a global topology difference statistical value is generated; Based on the received path sequence check value and the redundant operation check pair, the byte encoding of the received path sequence check value and the backup path sequence check value in the redundant operation check pair is extracted byte by byte, the byte encoding difference value is determined, and the total number of difference bits is counted to generate a path sequence difference statistical value; Based on the global topology difference statistics and the path sequence difference statistics, it is judged whether the global topology difference statistics and the path sequence difference statistics are equal to zero. If the global topology difference statistics and the path sequence difference statistics are both zero, it is confirmed that the synchronization consistency state is synchronization success. If the global topology difference statistics or the path sequence difference statistics is greater than zero, it is confirmed that the synchronization consistency state is synchronization failure, and a synchronization consistency confirmation state is obtained.

Images