A data synchronization method and system for a server cluster
By constructing a multi-column linearly independent matrix as the encoding matrix, the problem of low data synchronization efficiency in server clusters is solved, achieving efficient data synchronization in optical interconnect data centers, reducing the number of synchronization time slots and improving transmission efficiency.
Patent Information
- Application Number
- CN202111255797.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-27
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-10-27
AI Technical Summary
In server clusters, existing methods cannot effectively improve data synchronization efficiency because data distribution cannot be pre-arranged. This is especially true in optical interconnect data centers, where traditional data synchronization methods have low slot efficiency and transmission efficiency.
A multi-column linearly independent matrix is used as the encoding matrix. The encoding matrix is constructed by the control plane and multiplied with the synchronization data to generate encoded data. The missing data is then decoded and recovered at the cooperative node. The higher-order galois field is used to improve the encoding efficiency.
It effectively reduces the number of synchronization time slots, lowers transmission latency, improves transmission efficiency, adapts to different data distribution types, and meets the data synchronization needs of multi-node collaborative tasks.
Smart Images

Figure CN114185953B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of data transmission, and in particular, relates to a data synchronization method and system for a server cluster. Background Art
[0002] Due to the development of cloud storage and cloud computing, computing tasks are getting larger and larger. Tasks often need to be processed on large data sets scattered across different servers. Therefore, multiple nodes need to collaborate and a MapReduce-type framework is applied. In such tasks, it is important to synchronize distributed data in the nodes.
[0003] Since data is stored in different nodes in a dispersed manner, backup is usually used to prevent the risk of data loss and improve the efficiency of data synchronization, so each node already has part of the data. The data in different nodes is different. The traditional method is to broadcast all data to all nodes, and each node will select the data it lacks. For example, the patent "A method and system for synchronizing optical network unit information in a passive optical network" with publication number CN105515707A proposes that when the state of the optical network unit changes, the optical line terminal updates the information of the optical network unit in the optical network unit information library; the optical line terminal sends a status change prompt to the network management server; the network management server synchronizes the information of the optical network unit stored in the local database according to the status change prompt. The patent "A single-source multi-node data synchronization method and system" with publication number CN112445799A proposes to establish a change data table through a trigger, insert multiple change data into the change data table, read multiple change data through multiple nodes respectively, and select the data to be synchronized in the change data for synchronization. The solution of the present invention reads multiple change data in a change data table through multiple nodes respectively, and selects the data to be synchronized in the change data for synchronization, so as to generate only one trigger data of the data source. However, the time slot efficiency and transmission efficiency of this method are low.
[0004] In fog network data synchronization, which currently uses encoding methods, data distribution is pre-arranged to match a fixed data encoding method. However, in the server cluster of an optical interconnected data center, the data distribution of the server cluster cannot be pre-arranged, so the above method cannot effectively improve the data synchronization efficiency.
[0005] In order to improve the data synchronization efficiency of a server cluster, a data synchronization method and system for a server cluster are proposed. Summary of the invention
[0006] In order to solve the above problems, the present invention provides a data synchronization method and system for a server cluster.
[0007] A data encoding method for a server cluster of the present invention is characterized by comprising:
[0008] The control plane constructs a multi-column linearly independent matrix as a coding matrix according to the amount of data required to be synchronized and the amount of synchronization data missing in each coordinated node;
[0009] The task initiating node multiplies each row element of the encoding matrix with the corresponding synchronization data and then performs logical summation to obtain the encoded data.
[0010] Preferably, the amount of synchronization data missing in each collaborative node is any one or a combination of the amount of synchronization data missing in each collaborative node, or the maximum amount of synchronization data missing in each collaborative node, or the average amount of synchronization data missing in each collaborative node, obtained by the control plane by comparing the node data reported by each collaborative node with the required synchronization data.
[0011] Preferably, the step of constructing a plurality of linearly independent matrices as encoding matrices according to the amount of data to be synchronized and the amount of synchronized data missing in each coordinated node comprises the following steps:
[0012] The amount of data that needs to be synchronized is recorded as N, and the amount of synchronization data that is missing in each collaborative node is recorded as M;
[0013] Construct one or more N-column matrices with any M-column linear independence relationship;
[0014] Select an N-column matrix with the least number of rows as the encoding matrix.
[0015] Further preferably, the N-column matrix with any M-column linear independence relationship is calculated based on the parity check matrix in the error control coding, and the elements in the matrix are any items in the second-order galois field GF(2) or the high-order galois field GF(n). Note: Galois is translated into Galois in Chinese.
[0016] Preferably, the task initiating node multiplies each row element of the coding matrix by the corresponding synchronization data and then performs a logical sum to obtain the coding data, including the steps of:
[0017] The number of rows in the encoding matrix is denoted as s, and the matrix elements are denoted as a ij , 1≤i≤s, 1≤j≤N;
[0018] The task initiating node sends the required synchronization data (D1, ..., D N ) is multiplied with each row element of the encoding matrix and then logically summed, that is, Get s coded data (C1, ..., C s ).
[0019] A data decoding method for a server cluster of the present invention comprises:
[0020] The collaborative node constructs a recovery matrix based on the encoding matrix, the existing data in the node and the received encoding data;
[0021] Perform matrix transformation on the recovery matrix to obtain the solution for each missing data in the node;
[0022] The missing data in the node is obtained based on the solution of each missing data.
[0023] Preferably, the step of constructing a recovery matrix according to the coding matrix, existing data in the node and received coding data comprises the following steps:
[0024] The collaborative node multiplies each row element of the encoding matrix with the existing data in the corresponding node and then sums them (Galois domain addition) to obtain the sum of the existing data of the node;
[0025] Subtract the received coded data from the existing data of the node (Galois field subtraction) to obtain the partial decoded data, which is expressed as (C 1' , …, C S' ), where s represents the amount of encoded data;
[0026] The recovery matrix is constructed as a coefficient matrix of the partially decoded data.
[0027] Preferably, performing matrix transformation on the recovery matrix to obtain a solution for each missing data in the node includes:
[0028] The first m rows of the recovery matrix are transformed into an m×m diagonal matrix through matrix transformation, where m represents the number of synchronization data missing from the cooperative node;
[0029] The solutions corresponding to the m rows in the diagonal matrix form after the matrix transformation are the solutions for the m missing data in the nodes.
[0030] A data synchronization method for a server cluster of the present invention is characterized by comprising:
[0031] The control plane constructs a coding matrix according to the above data coding method and sends the coding matrix to the task initiating node and each collaborative node;
[0032] The task initiating node encodes the required synchronization data according to the above data encoding method to obtain encoded data;
[0033] The task initiating node sends the encoded data to each collaborative node;
[0034] Each collaborative node performs data decoding according to the above data decoding method to obtain the data missing in the node.
[0035] A data synchronization system for a server cluster, characterized by comprising: an optical interconnection data center;
[0036] Control plane;
[0037] Task initiation node;
[0038] Collaboration nodes;
[0039] Memory;
[0040] as well as
[0041] One or more programs, wherein the one or more programs are stored in a memory and configured to be executed by the control plane and / or the task initiating node and / or the coordination node, and the programs enable a computer to execute the above method.
[0042] The method and system of the present invention have the advantages that:
[0043] (1) The data that needs to be synchronized in multi-node collaborative tasks is encoded and then transmitted. Compared with the traditional technical method of broadcasting all synchronized data, it can effectively reduce the number of synchronization time slots and reduce transmission delay.
[0044] (2) Construct a multi-column linearly independent matrix as the encoding matrix. The linear independence of the matrix columns can effectively ensure that each node can directly decode its lost data through the encoded data, meet the needs of different data distribution types, effectively reduce the broadcast time slot, and improve transmission efficiency.
[0045] (3) A recoverable coding matrix is constructed in the high-order Galois domain, which can be recovered using less coded data, thereby improving the coding efficiency and further reducing the time slot. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 (a) is a schematic diagram of the synchronization structure in which the data in the POI of the coupler is not encoded.
[0047] Figure 1 (b) is a schematic diagram of the synchronization structure for encoding data in the POI based on the coupler.
[0048] Figure 2 The invention is a flow chart of a data encoding method for a server cluster.
[0049] Figure 3 It is a schematic diagram of data distribution and encoding strategy.
[0050] Figure 4 The invention is a flow chart of a data decoding method of a server cluster.
[0051] FIG5(a) is a diagram of the data recovery structure of Node1.
[0052] Figure 5(b) is a diagram of the data recovery structure of Node2.
[0053] Figure 5(c) is the data recovery structure diagram of Node3.
[0054] Figure 6 The invention is a flow chart of a data synchronization method for a server cluster.
[0055] Figure 7 The present invention is a structural diagram of a data synchronization system of a server cluster. DETAILED DESCRIPTION
[0056] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0057] In coupler-based POIs, data synchronization can be achieved through broadcasting. Figure 1 (a) shows the traditional synchronization method, where the task initiating node broadcasts six data one by one, requiring a total of 6 time slots (the same as the number of data). Figure 1 In (b), the logical encoding method is applied to compress the six data into two coded data ( and ), nodes 1 and 4 can be obtained from Node 2 and Node 3 can recover data3 from Nodes 5 and 6 can recover data2 from Node 1 and Node 2 can recover data6 from Data4⊕Data5⊕Data6 because they have data4 and data5. Node 3, Node 4, and Node 5 can recover data5 from Data4→Data5→Data6 because they have data4 and data6. Node 6 can recover data4 from Data4⊕Data5⊕Data6 because they have data5 and data6.
[0058] exist Figure 1In (b), data 1, data 2 and data 3 are encoded and broadcast in the first time slot, while data 4, data 5 and data 6 are encoded and broadcast in the second time slot. Compared with the traditional synchronization method, this data encoding method reduces the number of time slots from 6 to 2. In the data center, the data distribution depends on the previous network behavior, which is chaotic and diverse. The difference in data distribution will lead to different encoding methods. Therefore, the present invention proposes a data synchronization method and system for a server cluster, which can use the best encoding to execute the data synchronization process in any multi-node collaborative task in the optical interconnection center network, reduce the number of synchronization time slots, and improve the efficiency of optical interconnection data synchronization.
[0059] An embodiment of a data encoding method for a server cluster of the present invention is shown in the flowchart as follows: Figure 2 As shown, it is characterized by comprising:
[0060] The control plane constructs a multi-column linearly independent matrix as a coding matrix according to the amount of data required to be synchronized and the amount of synchronization data missing in each coordinated node;
[0061] The task initiating node multiplies each row element of the encoding matrix with the corresponding synchronization data and then performs logical summation to obtain the encoded data.
[0062] Preferably, the amount of synchronization data missing in each collaborative node is the amount of synchronization data missing in each collaborative node obtained by the control plane by comparing the intra-node data reported by each collaborative node with the required synchronization data, or the maximum amount of synchronization data missing in each collaborative node, or the average amount of synchronization data missing in each collaborative node, or any one or more combinations thereof. In this embodiment, there are two key parameters in data synchronization, the first is the data number to be synchronized (i.e., the amount of data required to be synchronized by the task initiating node, denoted as N), and the second is the number of lost data (i.e., the amount of synchronization data missing in each collaborative node). The amount of synchronization data missing in each collaborative node may be different. The control plane obtains the amount of synchronization data missing in each collaborative node by comparing the intra-node data reported by each collaborative node with the required synchronization data of the task initiating node. Any one or combination of the maximum amount of synchronization data missing in each collaborative node or the average amount of synchronization data missing in each collaborative node is used as an indicator of the amount of data missing in the collaborative node, denoted as M. In this embodiment, the maximum amount of synchronization data missing in each collaborative node is used as an indicator of the amount of data missing in the collaborative node, that is, in this task, all participating nodes only need M data at most to achieve synchronization.
[0063] The number of broadcast time slots required for data synchronization (i.e., the number of synchronization time slots) is represented by the variable s. The task initiating node distributes the N data to be synchronized to all nodes in the s time slots. If the data is broadcast once without encoding, then s = N. In this embodiment, a certain collaborative node lacks M data. Therefore, it requires at least M time slots to obtain the missing synchronization data. Thus, s ≥ M. Also, because s < N under the encoding strategy, the synchronization time slot s should satisfy M ≤ s < N. The purpose of the optimal encoding method is to minimize the time slot S.
[0064] Preferably, constructing a multi-column linearly independent matrix as the encoding matrix according to the amount of data to be synchronized and the amount of missing synchronization data in each collaborative node includes the steps of:
[0065] The amount of data to be synchronized is denoted as N, and the amount of missing synchronization data in each collaborative node is denoted as M;
[0066] Construct one or more N-column matrices with any M-column linearly independent relationships;
[0067] Select an N-column matrix with the fewest number of rows as the encoding matrix.
[0068] In this embodiment, as Figure 3 shown, the task initiating node ( Figure 3 denoted as Source Node in
[0069] ) has 8 data to be synchronized, that is, the total number of data N = 8. Each collaborative node (Node1 to Nodek) lacks 3 data, that is, the index M of the amount of missing data in the collaborative node is 3. If any M columns in the encoding matrix are linearly independent, then the subsequent inverse recovery matrix is a full-rank matrix, and each possible combination of the missing data can be effectively recovered. Therefore, the current encoding matrix should be an 8-column matrix with any 3 columns linearly independent. At this time, there are multiple candidate matrices, and one of the candidate matrices is kbinary codewords of length n, so that every pair of distinct code blocks differs in at least d positions, and if the code is linear, is completely specified by the parity check matrix H, which is a (n–k)×n binary matrix in which any d–1 columns are linearly independent. In the binary galois field GF(2), any parity check matrix H for a binary block code (n, k, d) can be used as an M-th order retrievable coding matrix (N, S, M) with parameters N = n, S = n–k, and M = d–1. For example, a parity check matrix H for a binary block code (8, 4, 4) is Any three columns in this matrix are linearly independent, so this 4×8 matrix can be used as a third-order reversible coding matrix (8, 4, 3).
[0070] Further preferably, the N-column matrix with arbitrary M-column linear independence is calculated based on the parity check matrix in the error control coding, and the elements in the matrix are any items in the second-order galois field GF(2) or the high-order galois field GF(n).
[0071] In a preferred embodiment, the field is extended to a higher-order galois field GF(n), and the elements in the encoding matrix can be 0, 1,…, n-1. When performing arithmetic operations in GF(n), the binary logical addition (⊕) should be changed to addition or subtraction operations in GF(n). Compared with GF(2), the column vectors in the higher-order galois field GF(n) may be linearly independent, so the matrix in GF(n) can achieve M-order recoverability with fewer rows. In GF(2), a six-column 2nd-order recoverable matrix requires at least three rows, but in GF(5), only two rows are required at least. An optional matrix is
[0072] In addition, the operation process of the high-order galois field GF(n) is different from that of GF(2). In one embodiment, there are six data that need to be synchronized (N=6), and each node is missing two data (M=2). Nodex loses Data2 and Data5, and Nodey loses Data1 and Data6. The 2nd-order recoverable coding matrix is selected as Each data contains six bits. In the task initiation node, the original binary data is mapped to level 5 data, six bits are mapped to three level 5 symbols, and then the level 5 data is encoded according to the encoding matrix. Five is a prime number, so if the calculation result is not in {0, 1, 2, 3, 4}, the calculation result should be MOD5 (MOD is modular calculation).
[0073] In the higher order galois domain, more coefficients can be used for encoding, so less coded data can be used for recovery, which improves the coding efficiency. In the higher order galois domain, the required time slot S will be reduced.
[0074] Preferably, the task initiating node multiplies each row element of the coding matrix by the corresponding synchronization data and then performs a logical sum to obtain the coding data, including the steps of:
[0075] The number of rows in the encoding matrix is denoted as s, and the matrix elements are denoted as a ij , 1≤i≤s, 1≤j≤N;
[0076] The task initiating node sends the required synchronization data (D1, ..., D N ) is multiplied with each row element of the encoding matrix and then logically summed, that is, Get s coded data (C1, ..., C s ). It should be noted that the plus sign in equation (1) represents the Galois field addition.
[0077] In this embodiment, the N data to be synchronized are represented as original data (D1, ..., D N ). Each row of the coding matrix corresponds to a time slot, and each column corresponds to data. Taking the second-order galois field GF(2) as an example, the value of each element is "1" or "0", and the result of the logical sum with each data to be synchronized indicates whether the data is included in the time slot or not. The coded data corresponding to each time slot is represented by (C1, ..., C s ). According to the coding matrix The number of rows of the coding matrix (number of synchronization time slots) s = 4. The task initiating node merges the data into 4 coded data (C1, C2, C3, C4).
[0078] In a preferred embodiment, when the elements in the matrix are high-order galois field GF(n) values, a high-order modulation transmission method is used instead of binary transmission. In this embodiment, when encoding is performed in the high-order galois field GF(n), the binary data (data in GF(2)) should be mapped to n-level data (data in GF(n)) before encoding. Accordingly, each level 3 symbol, level 4 symbol, level 5 symbol and level 9 symbol carries 1.5 bits, 2 bits, 2 bits and 3 bits respectively. The mode "22" in GF(3), the mode "4" in GF(5) and the mode "8" in GF(9) cannot be used after mapping, but they may appear after encoding. After encoding, the encoded data will be broadcast to the collaborative nodes. The n-level encoded data should be mapped again to match the transmission format. Mapping may result in bandwidth loss. If the transmission format is a traditional binary format, the encoded n-level data should be mapped to binary data again before broadcasting it to the collaborative nodes. Level 3 data, level 5 and level 9 data will suffer bandwidth loss due to mapping. In order to reduce the loss, every three 3-level data (27 modes) should be mapped to five binary data (32 modes), every three 5-level data (125 modes) should be mapped to seven binary data (128 modes), and each 9-level data (9 modes) should be mapped to two 3-level data (9 modes) and then mapped to binary data (32 modes). Each 4-level data (4 modes) should be mapped to two binary data (4 modes). Considering that each binary symbol, 3-level symbol, 4-level symbol, 5-level symbol and 9-level symbol carry 1 bit, 1.5 bits, 2 bits, 2 bits and 3 bits, each mapped binary symbol carries 1 bit, 0.9 (1.5×3 / 5) bit, 1 (2×1 / 2) bit, 0.86 (2×3 / 7) bit and 0.9 (same as the 3-level case) bit.
[0079] High-order modulation is applied to replace binary transmission. For example, the most commonly used high-order modulation in optical interconnect data centers is pulse amplitude modulation (PAM). If n-level data is transmitted directly in PAMn, the mapping bandwidth loss can be avoided. However, when n becomes larger, the PAMn signal will be severely degraded. Viable PAM formats include PAM3 and PAM4, which are reported to have a receiving sensitivity of 5dB and 8dB worse than binary signals. If the encoded data is broadcast in PAM3, the 3-level data does not need to be mapped. Every three binary data (8 modes) should be mapped to two 3-level data (9 modes), every three 4-level data (64 modes) should be mapped to four 3-level data (81 modes), every 2 5-level data (25 modes) should be mapped to three 3-level data (27 modes), and every 9-level data (9 modes) should be mapped to two 3-level data (9 modes). Considering that each binary symbol, 3-level symbol, 4-level symbol, 5-level symbol, and 9-level symbol carry 1 bit, 1.5 bits, 2 bits, 2 bits, and 3 bits, each PAM3 symbol carries 1.5 (1×3 / 2) bits, 1.5 (1.5×1 / 1) bits, 1.5 (2×3 / 4) bits, 1.33 (2×2 / 3) bits, and 1.5 (3×1 / 2) bits. If the encoded data is broadcast in PAM4, the mapping method is the same as that in binary transmission. When the transmission format is binary or PAM4, the mapping bandwidth efficiency of data in GF(3), GF(4), GF(5), and GF(9) is 90%, 100%, 86%, and 90%, respectively. When the transmission format is PAM3, the mapping bandwidth efficiency becomes 100%, 100%, 88%, and 100%, respectively. The mapping bandwidth loss will not exceed 15%.
[0080] A data decoding method for a server cluster according to an embodiment of the present invention is shown in the flowchart as follows: Figure 4 As shown, including:
[0081] The collaborative node constructs a recovery matrix based on the encoding matrix, the existing data in the node and the received encoding data;
[0082] Perform matrix transformation on the recovery matrix to obtain the solution for each missing data in the node;
[0083] The missing data in the node is obtained based on the solution of each missing data.
[0084] Preferably, the step of constructing a recovery matrix according to the coding matrix, existing data in the node and received coding data comprises the following steps:
[0085] The collaborative node multiplies each row element of the encoding matrix with the existing data in the corresponding node and then sums them (Galois domain addition) to obtain the sum of the existing data of the node;
[0086] Subtract the received coded data (Galois field subtraction) from the existing data of the node and get the partial decoded data, expressed as (C1', ..., C S '), where s represents the amount of encoded data;
[0087] The recovery matrix is constructed as a coefficient matrix of the partially decoded data.
[0088] In this embodiment, in this embodiment, for the loss of Dk1, Dk2, ..., Dk m ,(k1,k2,…,k m ∈{1,2,…,N}) and already have Dn1,Dn2,…,Dn (N-m) ,(n1,n2,…,n (N-m) ∈{1,2,…,N}), the cooperative node first multiplies each row element of the encoding matrix with the existing data in the corresponding node and then sums them (Galois field addition) to obtain the sum of the existing data of the node, and then subtracts the received encoded data (Galois subtraction) from the sum of the existing data of the node to obtain partial decoded data, which is expressed as (C1',…,C S '), as shown in equation (2). For each node, the decoded data only contains the lost data. Partially decoded data (C1', ..., C S ') is an S×m matrix, which is the recovery matrix R.
[0089]
[0090]
[0091] …
[0092]
[0093] For example, Node1, Node2, and Node3 are cooperative nodes. Each node lacks 3 synchronization data, that is, m=3. The coded data received by each cooperative node is The known data in the node is logically summed with each encoded data to obtain partial decoded data (C1', ..., C S '), partially decoded data (C1', ..., C S ') constitutes a 4×3 recovery matrix.
[0094] Preferably, performing matrix transformation on the recovery matrix to obtain a solution for each missing data in the node includes:
[0095] The first m rows of the recovery matrix are transformed into an m×m diagonal matrix through matrix transformation, where m represents the number of synchronization data missing from the cooperative node;
[0096] The solutions corresponding to the m rows in the diagonal matrix form after the matrix transformation are the solutions for the m missing data in the nodes.
[0097] In this embodiment, the coded data in the above embodiment is The recovery matrix is composed of the partially decoded data. The first 3 rows of each recovery matrix are converted into a 3×3 diagonal feature through matrix transformation. The solution corresponding to the first 3 rows is the solution for each missing data in the node. The decoding process is shown in Figure 5(a)(b)(c).
[0098] A data synchronization method for a server cluster according to an embodiment of the present invention is shown in the flowchart as follows: Figure 6 As shown, including:
[0099] The control plane constructs a coding matrix according to the above data coding method and sends the coding matrix to the task initiating node and each collaborative node;
[0100] The task initiating node encodes the required synchronization data according to the above data encoding method to obtain encoded data;
[0101] The task initiating node sends the encoded data to each collaborative node;
[0102] Each collaborative node performs data decoding according to the above data decoding method to obtain the data missing in the node.
[0103] In this embodiment, Figure 1 Taking the data in the coupler-based POI as an example, the control plane constructs a 2nd-order recoverable coding matrix according to the data coding method of the above embodiment. The encoding matrix is sent to the task initiating node and each collaborative node; the task initiating node encodes the synchronization data (D1, ..., D6) according to the data encoding method of the above embodiment to obtain the encoded data The task initiating node sends the encoded data to each collaborative node, such as node 1. Node 1 has D1, D2, D4, D5, and lacks D3 and D6. Node 1 decodes the data according to the data decoding method of the above embodiment to obtain partial decoded data. The recovery matrix R is calculated as The recovery matrix R is a diagonal matrix, which can directly obtain the solutions of the missing D3 and D6, thereby achieving data synchronization.
[0104] A data synchronization system for a server cluster according to an embodiment of the present invention is shown in FIG. Figure 6 As shown, it is characterized by comprising:
[0105] Control plane;
[0106] Task initiation node;
[0107] Collaboration nodes;
[0108] Memory;
[0109] as well as
[0110] One or more programs, wherein the one or more programs are stored in a memory and configured to be executed by the control plane and / or the task initiating node and / or the coordination node, and the programs enable a computer to execute the above method.
[0111] Of course, those skilled in the art should realize that the above embodiments are only used to illustrate the present invention, and are not intended to limit the present invention. As long as they are within the scope of the present invention, any changes or modifications to the above embodiments will fall within the protection scope of the present invention.
Claims
1. A data encoding method for a server cluster, It is characterized in that include: The control plane constructs a plurality of linearly independent matrices as a coding matrix according to the amount of data required to be synchronized and the amount of synchronization data missing in each collaborative node; the method of constructing a plurality of linearly independent matrices as a coding matrix according to the amount of data required to be synchronized and the amount of synchronization data missing in each collaborative node comprises the following steps: the amount of data required to be synchronized is recorded as N, and the amount of synchronization data missing in each collaborative node is recorded as M; Construct one or more N-column matrices with any M-column linear independence relationship; Select an N-column matrix with the least number of rows as the encoding matrix; The N-column matrix with any M-column linear independence relationship is calculated based on the parity check matrix in the error control coding, and the elements in the matrix are any items in the second-order galois field GF(2) or the high-order galois field GF(n); The task initiating node multiplies each row element of the encoding matrix with the corresponding synchronization data and then performs logical summation to obtain the encoded data.
2. The data encoding method of the server cluster according to claim 1, It is characterized in that The amount of synchronization data missing in each collaborative node is any one or more combinations of the amount of synchronization data missing in each collaborative node, the maximum amount of synchronization data missing in each collaborative node, or the average amount of synchronization data missing in each collaborative node, obtained by the control plane by comparing the node data reported by each collaborative node with the required synchronization data.
3. The data encoding method of the server cluster according to claim 1, It is characterized in that The task initiating node multiplies each row element of the coding matrix by the corresponding synchronization data and then performs a logical sum to obtain the coding data, including the steps of: The number of rows in the encoding matrix is denoted as s, and the matrix elements are denoted as a ij , 1≤i≤s, 1≤j≤N; The task initiating node will synchronize the required data (D 1 , …, D N ) is multiplied with each row element of the encoding matrix and then logically summed, that is, Get s coded data (C 1 , …, C s ).
4. A data decoding method for a server cluster, It is characterized in that include: The collaborative node constructs a recovery matrix based on the encoding matrix, the existing data in the node and the received encoding data; The recovery matrix is constructed according to the coding matrix, the existing data in the node and the received coding data, including the steps of: the collaborative node multiplies each row element of the coding matrix by the existing data in the corresponding node and then sums them to obtain the sum of the existing data of the node; the received coding data minus the sum of the existing data of the node to obtain partial decoded data, which is expressed as (C 1 ', ..., C S '), where s represents the amount of coded data; constructing a recovery matrix as a coefficient matrix of partially decoded data; Perform matrix transformation on the recovery matrix to obtain the solution for each missing data in the node; The missing data in the node is obtained based on the solution of each missing data.
5. The data decoding method of the server cluster according to claim 4, It is characterized in that The matrix transformation of the recovery matrix to obtain a solution for each missing data in the node includes: The first m rows of the recovery matrix are transformed into an m×m diagonal matrix through matrix transformation, where m represents the number of synchronization data missing from the cooperative node; The solutions corresponding to the m rows in the diagonal matrix form after the matrix transformation are the solutions for the m missing data in the nodes.
6. A data synchronization method for a server cluster, It is characterized in that include: The control plane constructs a coding matrix according to the method as claimed in claim 1 and sends the coding matrix to the task initiating node and each collaborative node; The task initiating node encodes the required synchronization data according to the method as claimed in claim 3 to obtain encoded data; The task initiating node sends the encoded data to each collaborative node; Each cooperative node performs data decoding according to the method as claimed in claim 4 to obtain the data missing in the node.
7. A data synchronization system for a server cluster, Features include: Control plane; Task initiation node; Collaboration nodes; Memory; as well as One or more programs, wherein the one or more programs are stored in a memory and are configured to be executed by the control plane and / or the task initiating node and / or the collaborative node, and the programs enable a computer to execute the method according to any one of claims 1 to 6.
