Piggyback coding construction with local repair characteristic and fault node repair method

By constructing the PiggyBacking coding with local repair characteristics, it solves the problems of repairing bandwidth, high repairs and high complexity, and achieve the effect of low repair bandwidth, low repair and low decoding complexity.

CN119376997APending Publication Date: 2025-01-28CHANGAN UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411221028.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2025-01-28

AI Technical Summary

Technical Problem

The existing Piggybacking encoding technology fixes problems such as large bandwidth, high repair and high complexity.

Method used

By constructing the PiggyBacking encoding with local repair characteristics, it specifically includes a distributed storage system encoding model with the number of generic bands F-1, and is embedded in the sub-band through the PiggyBack function to reduce the repair bandwidth and repair. At the same time, simplify the simplification of simplifying Decoding process.

Benefits of technology

The low repair bandwidth, low repair and low decoding complexity are achieved, especially with the increase of coding parameters, the average repair bandwidth rate is close to the MSR code, and it gradually approaches 0.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119376997A_ABST
    Figure CN119376997A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of computers, and discloses a Piggyback coding construction with a local repair characteristic and a fault node repair method. The construction method comprises the following steps: firstly, generating a distributed storage system coding model with a plurality of sub-strips, and expanding the sub-strips to form expanded sub-strips; and then, embedding the sub-stripe information data block into a check set, and copying the check data block to the extended sub-stripe to obtain a Piggyback code with a local repair characteristic. In addition, the invention further discloses an improved Piggybacking coding structure with a local repair characteristic, and the bandwidth repair overhead is further reduced by limiting coding parameters and fully embedding check nodes. The two codes constructed by the method have the optimal repair bandwidth overhead in both information nodes and check nodes under most coding parameters, and the repair degree and the repair complexity performance of fault nodes are also remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer distributed storage coding structure, and in particular relates to a piggybacking coding structure with local repair characteristics and a fault node repair method. Background Art

[0002] Distributed Storage Systems (DSSs), which have the advantages of low cost, good scalability, and easy management, have significant advantages and application prospects in solving the explosive growth of massive data storage. However, DSSs will inevitably experience storage node failures. In order to ensure the reliability of data storage, DSSs mainly use erasure coding strategies to ensure data fault tolerance. Compared with replication strategies, erasure coding has lower storage overhead under the same fault tolerance. However, repairing a failed node requires downloading the entire file size, and the repair bandwidth overhead is too high.

[0003] In view of the limitations of the above three-copy replication and erasure codes, Rashmi et al. first proposed Piggybacking coding in 2013. Based on MDS code, Piggybacking coding linearly combines the data blocks of some sub-strips and embeds them into other sub-strips, so that Piggybacking coding can overcome the defects of large storage overhead of replica replication and high bandwidth overhead of erasure code repair while retaining the advantages of MDS properties and high code rate. Research on Piggybacking coding in current distributed storage systems shows that MDS code decoding is required when repairing some faulty nodes, which will lead to large repair bandwidth, high repair degree and high replication degree. Therefore, constructing Piggybacking coding with low repair bandwidth, low repair degree and low complexity is the focus of research in today's distributed storage systems. Summary of the invention

[0004] The object of the present invention is to provide a Piggybacking coding structure with local repair characteristics and a faulty node repair method to solve the problems of large repair bandwidth, high repair degree and high complexity in the existing Piggybacking coding technology.

[0005] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0006] On the one hand, the present invention provides a method for constructing a piggybacking code with local repair characteristics, which specifically includes the following steps:

[0007] Step 1, generating a distributed storage system encoding model with a sub-strip number of f-1, specifically: assuming that the current distributed storage system encoding parameters are (n, k), n represents the code length, and k represents the number of information nodes;

[0008] The original information a=[a1,a2,…,a k ] are encoded into n data blocks a1, a2, ..., a k ,f1(a),…,f r (a), where f i (a) (1≤i≤r) represents the r check data blocks obtained by encoding, r=nk, and the n encoded data blocks are respectively stored on n nodes of the distributed storage system, forming a distributed storage system encoding model;

[0009] The distributed storage system coding model is traversed f-1 times to generate a distributed storage system coding model with f-1 sub-strips; each sub-strip constitutes a complete coding set, and the i-th sub-strip s i (1≤i≤f-1) contains the data block a i,1 ,a i,2 ,…,a i,k ,f1(a i ),…,f r (a i );

[0010] Step 2: Expand the number of sub-strips f-1 in step 1 to obtain an extended sub-strip s f At the same time, the data blocks of the previous sub-strips are linearly combined to generate a Piggyback function which is embedded into the subsequent sub-strips to construct a Piggybacking code with local repair characteristics and an improved Piggybacking code with local repair characteristics.

[0011] Furthermore, in step 2, the construction of the piggybacking code with local repair characteristics specifically includes the following steps:

[0012] For constructing Piggybacking coding with local repair characteristics, the number of sub-strips is set to f = m + 2. Assume i (1≤i≤f=m+2) represents the i-th sub-strip, where s i (1≤i≤f-1=m+1) constitutes a complete coding set, s m+2 is the extended sub-strip; the variable t=k+r(1-m) represents the extended sub-strip s m+2 Remove substrip The number of unfilled spaces after the data block is copied in the verification data;

[0013] Step a, copy the rm check data blocks in the sub-strip and sequentially place them into the first rm spaces of the extended sub-strip s m+2 in the order from top to bottom; at the same time, place the i-th (1 ≤ i < r) check data block of the sub-strip s1 into the (k + i + 1)-th space of the extended sub-strip s m+2 ; place the r-th check data block of the sub-strip s1 into the (k + 1)-th space of the extended sub-strip s m+2 ;

[0014] Step b, evenly divide the k information data blocks of the sub-strip s i (1 ≤ i ≤ m) into r parts and sequentially embed them into the r check nodes of the sub-strip s i+1 as the Piggyback function;

[0015] Step c, evenly divide the k information data blocks of the sub-strip s m+1 into t parts and sequentially embed them into the last t spaces of the extended sub-strip s m+2 .

[0016] Furthermore, in step 2, the construction of the improved Piggybacking coding with local repair characteristics specifically includes the following steps:

[0017] For the improved Piggybacking coding with local repair characteristics, set the number of sub-strips f = m + 1, and at the same time s m+1 is the extended sub-strip; the variable t = k + r(1 - m) represents the number of unfilled spaces in the extended sub-strip after copying the check data blocks in the sub-strip ; when m > r, the construction process is similar to the Piggybacking coding with local repair characteristics; first let t3 = t - mt l3 ;

[0018] Step A, copy the total rm check data blocks in the sub-strip and sequentially place them into the first rm spaces of the extended sub-strip s m+1 ;

[0019] Step B, evenly divide the k information data blocks of the sub-strip s i (1 ≤ i ≤ t3) into r + t h3 parts and embed them as the Piggyback function into the r check nodes of the sub-strip s i+1 and the t m+1 spaces of the extended sub-strip s h3 ; place the sub-strip s iThe k information data blocks where (t3 + 1 ≤ i < m) are evenly divided into r + t l3 parts and are embedded into the r check nodes of the sub-strip s i+1 and the t m+1 spaces of the extended sub-strip s l3 using the Piggyback function;

[0020] Step C: The k information data blocks of the sub-strip s m are evenly divided into r + t l3 parts and are embedded into the r check nodes of the sub-strip s1 and the t m+1 spaces of the extended sub-strip s l3 using the Piggyback function.

[0021] On the other hand, the present invention provides a method for repairing a failed node. The method is used to repair the Piggybacking code with local repair characteristics obtained by the Piggybacking coding construction method with local repair characteristics of the present invention, and the specific operations are as follows:

[0022] When a single node fails, find the data block lost by the failed node in the local repair table, and then download the other non-lost data blocks in the column where the lost data block is located and perform an exclusive OR operation to complete the repair of the data block; the repair methods for other lost data blocks are the same, and the repair of the failed node is completed.

[0023] The present invention has the following technical characteristics compared with the prior art:

[0024] (1) Compared with the existing Piggybacking codes, the two Piggybacking codes constructed by the present invention both satisfy the (n, k) property.

[0025] (2) For the construction of the present invention, both information nodes and check nodes have the optimal repair bandwidth overhead. In particular, as the coding parameters increase, the average repair bandwidth rate of the Piggybacking code with local repair characteristics approaches the MSR code and gradually approaches 0.

[0026] (3) For the repair degree of the construction of the present invention, when the number of check nodes is greater than 3, the repair degree of the Piggybacking code with local repair characteristics is lower than that of all existing Piggybacking codes.

[0027] (4) When repairing a failed node in the construction of the present invention, no multiplication operation is required, so it is far superior to the existing Piggybacking coding construction in terms of decoding complexity.

[0028] Appendix Explanation

[0029] Figure 1 Encoding models for distributed storage systems;

[0030] Figure 2 This is the structure diagram of the check node k+i in the Piggybacking encoding;

[0031] Figure 3 PCLR when (n, k) = (16, 13);

[0032] Figure 4 The allocation of k information data blocks of s1 when (n, k) = (16, 13) in PCLR;

[0033] Figure 5 The allocation of k information data blocks of s5 when (n, k) = (16, 13) in PCLR;

[0034] Figure 6 is the LRT when (n, k) = (16, 13) in PCLR;

[0035] Figure 7 Construct a graph of the minimum average repair bandwidth rate of information nodes for PCLR and existing Piggybacking coding as the check node r increases;

[0036] Figure 8 Construct a graph of the minimum average repair bandwidth rate of check nodes for PCLR and existing Piggybacking coding as the check node r increases;

[0037] Fig. 9 Comparison chart of average repair ratio of information nodes between PCLR and existing Piggybacking coding under different coding parameters (n, k);

[0038] Fig.10 Comparison chart of the average repair ratio of check nodes under different encoding parameters (n, k) between PCLR and existing Piggybacking encoding;

[0039] Fig.11 Construct a comparison table of the complexity of multiplication operations required for PCLR and existing Piggybacking encodings.

[0040] Fig.12 IPCLR when (n,k) = (16,13);

[0041] Fig.13 The allocation of k information data blocks of s4 when (n, k) = (16, 13) in IPCLR;

[0042] Fig.14 It is the LRT when (n,k)=(16,13) in IPCLR.

[0043] Fig.15 It is a three-dimensional comparison diagram of the minimum average repair bandwidth rate of information nodes for PCLR and IPCLR when the number of information nodes k and the number of check nodes r change;

[0044] Fig.16 It is a three-dimensional comparison diagram of the minimum average repair bandwidth rate of check nodes of PCLR and IPCLR when the number of information nodes k and the number of check nodes r change.

[0045] The present invention is further explained below with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] First, the technical terms appearing in the present invention are explained:

[0047] (1) Piggybacking coding: On multiple sub-strips of the basic MDS code, the Piggybacking coding embeds the linear combination of the data blocks of the previous sub-strip as the Piggyback function into the subsequent sub-strip, thereby reducing the MDS code decoding, increasing the Piggyback function decoding, and reducing the repair bandwidth overhead. In the Piggybacking coding framework, each column represents a sub-strip, and each sub-strip constitutes a complete coding set, which contains k information data blocks and r check data blocks. The structure of the check node k+i in the Piggybacking coding is as follows: Figure 2 As shown, variable f is the total number of sub-strips, vector a j =(a j1 ,,a j2 ,…,a jk )(1≤j≤f) represents the k information data blocks in the jth sub-strip, f i (a j )(1≤i≤r,1≤j≤f) is the i-th check data block generated by MDS coding the vector. j,i (a1,…,a j-1 )(2≤j≤f,1≤i≤r) represents the embedded Piggyback function.

[0048] (2) Piggyback function matrix: Define the Piggyback function matrix The number of information nodes k is mapped to the rows of the matrix, and the Piggyback function F j (1≤j≤rm+t) is mapped to the columns of the matrix, where t=k+r(1-m)(r≤t<2r). If P i,j =1 means that a data block in node i (1≤i≤k) exists in the Piggyback function F jIn the example, the XOR sum of all 1 elements in the jth column corresponding to the node data blocks is F j .

[0049] (3) M and N are composed of multiple adjustment matrices and supplementary matrices spliced ​​vertically. The construction rules of the adjustment matrix are different according to the parity of its order: when the order is odd, the adjustment matrix is ​​obtained by the inverse unit matrix through column swapping, that is, the first column is swapped with the middle column; when the order is even, the adjustment matrix directly uses the inverse unit matrix. The supplementary matrix is ​​formed by intercepting the first several rows of the corresponding adjustment matrix. The order of the adjustment matrix in the matrix M is equal to r, and the order of the adjustment matrix in the matrix N is equal to t. The 3rd order adjustment matrix g is shown below:

[0050]

[0051] (4) Localized Repair Table (LRT): The local repair table LRT is constructed as follows: the first row contains the check data blocks contained in the extended sub-strip from top to bottom. If a check data block is not placed in a certain position, the corresponding cell of the LRT row is left blank; the second row contains data blocks arranged in ascending order of the Piggyback function index. These data blocks have embedded Piggyback functions; the information data blocks associated with these Piggyback functions are listed in order in the corresponding columns.

[0052] In this embodiment, a method for constructing a Piggybacking code with a local repair feature is disclosed. The method expands the number of sub-strips of the Piggybacking framework, and then sets the Piggyback function F through the Piggyback function matrix to construct the Piggybacking code with a local repair feature. The specific steps are as follows:

[0053] Step 1, generating a distributed storage system coding model with the number of sub-strips being f-1, specifically: assuming that the current distributed storage system coding parameters are (n, k), n represents the code length, and k represents the number of information nodes.

[0054] The original information a=[a1,a2,…,a k ] are encoded into n data blocks a1, a2, ..., a k ,f1(a),…,f r (a), where f i (a) (1≤i≤r) represents the r check data blocks obtained by encoding, r=nk, and the n encoded data blocks are respectively stored on n nodes of the distributed storage system, forming a distributed storage system encoding model;

[0055] Traverse the encoding model of the distributed storage system f - 1 times to generate an encoding model of the distributed storage system with the number of sub - stripes being f - 1; each sub - stripe constitutes a complete encoding set, and the data blocks included in the i - th sub - stripe s i (1 ≤ i ≤ f - 1) are a i,1 , a i,2 , …, a i,k , f1(a i ), …, f r (a i ).

[0056] Step 2, expand the number of sub - stripes in Step 1 to obtain the extended sub - stripe s f , and at the same time, linearly combine the data blocks of the previous sub - stripes to generate the Piggyback function and embed it into the subsequent sub - stripes. Construct Piggybacking Codes with Locally Repair (PCLR) and Improved Piggybacking Codes with Locally Repair (IPCLR).

[0057] For constructing PCLR, set the number of sub - stripes f = m + 2, Let s i (1 ≤ i ≤ f = m + 2) represent the i - th sub - stripe, where s i (1 ≤ i ≤ f - 1 = m + 1) constitutes a complete encoding set, and s m+2 is the extended sub - stripe. The variable t = k + r(1 - m) represents the number of unfilled spaces in the extended sub - stripe after removing the replicated check data blocks in the sub - stripe .

[0058] (1) Copy the rm check data blocks in the sub - stripe and place them in the first rm spaces of the extended sub - stripe s m+2 in order from top to bottom. At the same time, place the i - th (1 ≤ i < r) check data block of the sub - stripe s1 into the (k + i + 1) - th space of the extended sub - stripe s m+2 . Place the r - th check data block of the sub - stripe s1 into the (k + 1) - th space of the extended sub - stripe s m+2 .

[0059] (2) Divide the k information data blocks of the sub - stripe s i (1 ≤ i ≤ m) into r equal parts as the Piggyback function and embed them into the r check nodes of the sub - stripe s i+1 .

[0060] (3) Sub-strip s m+1 The k information data blocks are divided into t parts as the Piggyback function and embedded into the extended sub-strip s m+2 In the last t spaces.

[0061] According to the above construction process, it can be found that there are a total of rm+t Piggyback functions. Starting from the first check data block of sub-strip s2, the embedded Piggyback function F is defined in the order from top to bottom and from left to right. j (1≤j≤rm+t). The matrix M contains rm columns, each r column is a group, and together with the t columns of the matrix N, it generates the Piggyback function set P = {p1,p2,…,p m ,p m+1}, |P|=m+1. i ={F (i-1)r+1 ,F (i-1)r+2 ,…,F ir}(1≤i≤m), p i The data block contained in all Piggyback functions is s i k information data blocks in p m+1 ={F rm+1 ,F rm+2 ,…,F rm+t}, p m+1 The data blocks contained in all Piggyback functions in are sub-strips s m+1 The k information data blocks in .

[0062] As an improved implementation of the present invention, in order to further reduce the repair bandwidth on the basis of satisfying the (n, k) property, the present invention also provides an IPCLR structure by restricting the coding parameters and fully embedding the check nodes. Steps 1 and 2 of the structure are the same as Steps 1 and 2 of the first method of the present invention above, and also include the following steps:

[0063] For constructing IPCLR, set the number of sub-strips f = m + 1, At the same time m+1 The variable t=k+r(1-m) represents the extended sub-strip. The number of unfilled spaces in the check data block after copying. When m>r, the general construction process is similar to PCLR. First, let t3=t-mt l3 .

[0064] (1) Sub-strip A total of rm check data blocks are copied in sequence and placed in the extended sub-strip s in order from top to bottom. m+1 In the first rm spaces.

[0065] (2) Divide the k information data blocks of the sub-strip s i (1 ≤ i ≤ t3) into r + t h3 parts and embed them as Piggyback functions into the r check nodes of the sub-strip s i+1 and the t m+1 spaces of the extended sub-strip s h3 . Divide the k information data blocks of the sub-strip s i (t3 + 1 ≤ i < m) into r + t l3 parts and embed them as Piggyback functions into the r check nodes of the sub-strip s i+1 and the t m+1 spaces of the extended sub-strip s l3 . Divide the k information data blocks of the sub-strip s m into r + t l3 parts and embed them as Piggyback functions into the r check nodes of the sub-strip s1 and the t m+1 spaces of the extended sub-strip s l3 .

[0066] Similarly, there are a total of rm + t Piggyback functions. The order of the adjustment matrix in matrix M is r, and the order of the adjustment matrix in matrix N is t. The difference is that at this time the Piggyback function set is P = {p1, p2, …, p m}, |P| = m.

[0067] When t l3 ≠ 0,

[0068]

[0069] When t l3 = 0,

[0070]

[0071] Among them, the data blocks included in all Piggyback functions in p i (2 ≤ i ≤ m) are the k information data blocks in the sub-strip s i-1 , and the data blocks included in all Piggyback functions in p1 are the k information data blocks in the sub-strip s m . At the same time, in p i (1 ≤ i ≤ m), if there is an element 1 of F j in matrix N, then the elements in the other F i corresponding rows in p j are 0.

[0072] When IPCLR is under other parameter conditions, its general structure is the same as when m > r. The main purpose is to ensure that the IPCLR encoding structure can still satisfy the (n,k) property under different parameter settings. Therefore, only the specific construction method of the Piggyback function matrix in the general structure under different parameter conditions is given. Case 1: When m = r, the only difference from m > r is that when constructing the Piggyback function matrix, the first r elements of F rm+1 are 1 and the rest are 0; Case 2: When r / 2 < m < r, the difference from m > r is that when constructing the Piggyback function matrix, if F j , F j′ ∈p i (j < j′) and F j , F j′ is the column corresponding to the matrix N, then the column corresponding to F j contains r 1s, and the other p i′ Similarly, the positions of 1s are determined according to the subscript order of F j .

[0073] The present invention discloses the PCLR construction and the IPCLR construction process. Specifically, PCLR embeds the sub-strip information data block into the parity set by setting appropriate sub-strip numbers and expanding the sub-strip, and copies the parity data block to the expanded sub-strip. At the same time, in order to further reduce the repair bandwidth on the basis of satisfying the (n,k) property, by restricting the coding parameters and making full use of the embedding of the parity nodes, an improved Piggybacking coding construction with local repair characteristics is also given. Compared with the existing Piggybacking coding, the PCLR and IPCLR constructed by the present invention have the optimal repair bandwidth overhead for both information nodes and parity nodes under most coding parameters, and the repair degree and repair complexity performance of the failed nodes are also significantly improved. In particular, as the coding parameters increase, the average repair bandwidth rate of PCLR approaches the MSR code and gradually approaches 0; in terms of the repair degree, when the number of parity nodes is greater than 3, the repair degree of PCLR is lower than that of all existing Piggybacking codings; in terms of the repair complexity, PCLR does not require multiplication operations, and its repair complexity performance is far superior to that of the existing Piggybacking codings.

[0074] The following gives specific embodiments and also analyzes the repair method when a node fails.

[0075] Embodiment 1

[0076] In the PCLR construction, when (n,k) = (16,13), through steps 1 and 2 in the method of the present invention above, we can obtain Figure 3The final encoding result. The matrix P is as follows. According to the matrix P, all Piggyback functions F can be obtained. i The information data block contained in the Figure 4 The k information data blocks of s1 are shown in Figure 1, i.e., p1 = {F1, F2, F3}, F1 = a2 + a5 + a8 + a 11 、F2=a3+a6+a9+a 12 、F1=a1+a4+a7+a 10 +a 13 . Figure 5 The k information data blocks of s5 are shown in the figure, that is, p5 = {F 13 ,F 14 ,F 15 ,F 16}, F 13 =e4+e8+e 12 、F 14 =e3+e7+e 11 、F 15 =e2+e6+e 10 、F 16 =e1+e5+e9+e 13 .

[0077]

[0078] Faulty node repair:

[0079] like Figure 6 Table 1 shows the LRT of PCLR when (n, k) = (16, 13). When node i (1≤i≤k+r) fails, it only needs to find the corresponding lost data block in the LRT, and then download the other non-lost data blocks in the column where the lost data block is located to complete the repair. For example, in (n, k) = (16, 13), information node 1 fails, and the lost data blocks are marked in dark gray in the LRT. To repair data block a1, it is only necessary to download the non-lost data blocks f3(b), f3(b)+F3, a4, a7, a1 in the column where a1 is located. 10 and a 13 , the repair is completed. The repair method for other lost data blocks is the same. The light gray in the LRT is the data block lost when the check node 14 fails. The repair method is also to download the data blocks that are not lost in this column.

[0080] The repair bandwidth of node 1 obtained by repairing the faulty node is The repair bandwidth of node 14 is Much lower than the bandwidth of MDS code decoding and repair. Figure 7 and Figure 8It shows that the PCLR constructed by the present invention is superior to the existing Piggybacking coding in terms of the minimum average repair bandwidth rate. Figure 7 and Figure 8 It can be found that, for both information nodes and check nodes, the minimum average repair bandwidth rate decreases as r increases under two different code rates, and its repair performance gradually approaches the repair performance of MSR code. Fig. 9 and Fig.10 A comparison chart of the average repair ratio of information nodes and check nodes between PCLR and existing Piggybacking coding is constructed for the present invention. It can be found that when the check node r=3, the average repair ratio of information nodes and check nodes of PCLR has advantages over the existing Piggybacking coding. When the number of check nodes increases to r=4, the average repair ratio of information nodes and check nodes of PCLR is significantly better than that of existing Piggybacking coding. Fig.11 Table 5 shows a comparison of the number of multiplication operations required for encoding, information node repair, and check node repair between the existing Piggybacking code and the PCLR constructed by the present invention. It can be found that the coding complexity of each Piggybacking code construction depends on the number of sub-strips, which is not analyzed in detail here. However, in terms of repair complexity, the PCLR constructed by the present invention does not require multiplication operations and is significantly better than the existing Piggybacking code.

[0081] Example 2

[0082] In the IPCLR structure, through the steps 1 and 2 of the method of the present invention, it is possible to obtain Fig.12 Final encoding result. For different encoding parameters, the Piggyback function matrix is ​​different, and the resulting Piggyback function is different. Taking (n, k) = (16, 13) as an example, t l3 =t h3 =1, t3=0. p1={F1,F2,F3,F 13}, F 13 The elements in the 4th, 8th and 12th rows of the matrix N are 1, so the elements in the 4th, 8th and 12th rows of F1, F2 and F3 are 0, and the elements in the other rows are 0. i Similarly, the following changing elements of matrix P are marked in light grey. Fig.13 This is the allocation of k information data blocks of s4.

[0083]

[0084] Faulty node repair:

[0085] The repair process is similar to that of PCLR. You only need to find the column where the lost data block is located and then download other non-lost data blocks in the column to complete the repair. Fig.14 Table 2 shown is the LRT of the IPCLR when (n, k) = (16, 13), wherein dark gray indicates the data blocks lost by the faulty node 1, and light gray indicates the data blocks lost by the faulty node 14.

[0086] Can get Compared with PCLR, the repair bandwidth overhead of the faulty node is reduced. Fig.15 and Fig.16 The three-dimensional variation diagram of the minimum average repair bandwidth rate of PCLR and IPCLR is given, which can more intuitively show that the minimum average repair bandwidth rate shows a decreasing trend with the number of information nodes k and the number of check nodes r. It can also reflect the performance advantage of IPCLR over PCLR in repair bandwidth overhead.

Claims

1. A method for constructing a piggybacking code with local repair characteristics, characterized in that: The specific steps include: Step 1, generating a distributed storage system encoding model with a sub-strip number of f-1, specifically: assuming that the current distributed storage system encoding parameters are (n, k), n represents the code length, and k represents the number of information nodes; The original information a=[a1,a2,…,a k ] are encoded into n data blocks a1, a2, ..., a k ,f1(a),…,f r (a), where f i (a) (1≤i≤r) represents the r check data blocks obtained by encoding, r=nk, and the n encoded data blocks are respectively stored on n nodes of the distributed storage system, forming a distributed storage system encoding model; The distributed storage system coding model is traversed f-1 times to generate a distributed storage system coding model with f-1 sub-strips; each sub-strip constitutes a complete coding set, and the i-th sub-strip s i (1≤i≤f-1) contains the data block a i,1 ,a i,2 ,…,a i,k ,f1(a i ),…,f r (a i ); Step 2: Expand the number of sub-strips f-1 in step 1 to obtain an extended sub-strip s f At the same time, the data blocks of the previous sub-strips are linearly combined to generate a Piggyback function which is embedded into the subsequent sub-strips to construct a Piggybacking code with local repair characteristics and an improved Piggybacking code with local repair characteristics.

2. The method for constructing a piggybacking code with a local repair characteristic according to claim 1, wherein in step 2, constructing the piggybacking code with a local repair characteristic specifically comprises the following steps: For constructing Piggybacking coding with local repair characteristics, the number of sub-strips is set to f = m + 2. Assume i (1≤i≤f=m+2) represents the i-th sub-strip, where s i (1≤i≤f-1=m+1) constitutes a complete coding set, s m+2 is the extended sub-strip; the variable t=k+r(1-m) represents the extended sub-strip s m+2 Remove substrip The number of unfilled spaces after the data block is copied in the verification data; Step a, copy the rm check data blocks in the sub-strip and sequentially place them into the first rm spaces of the extended sub-strip s m+2 in the order from top to bottom; at the same time, place the i-th (1 ≤ i < r) check data block of the sub-strip s1 into the (k + i + 1)-th space of the extended sub-strip s m+2 ; place the r-th check data block of the sub-strip s1 into the (k + 1)-th space of the extended sub-strip s m+2 . Step b: Sub-strip s i The k information data blocks (1≤i≤m) are equally divided into r parts and embedded into the sub-strip s in sequence as the Piggyback function. i+1 Among the r check nodes; Step c: Sub-strip s m+1 The k information data blocks are divided into t parts as Piggyback functions and embedded into the extended sub-strip s in sequence. m+2 In the last t spaces.

3. The method for constructing a piggybacking code with local repair characteristics according to claim 1, wherein in step 2, constructing the improved piggybacking code with local repair characteristics specifically comprises the following steps: For the improved Piggybacking coding with local repair characteristics, the number of sub-strips is set to f = m + 1. At the same time m+1 is the extended sub-strip; the variable t=k+r(1-m) represents the extended sub-strip with the sub-strip removed The number of unfilled spaces after the data block is copied in the checksum. When m>r, the construction process is similar to the Piggybacking code with local repair characteristics. First, let t3=t-mt l3 ; Step A: Sub-strip A total of rm check data blocks are copied and placed in the extended sub-strip s in order from top to bottom. m+1 In the first rm spaces; Step B, divide the k information data blocks of the sub-strip s i (1 ≤ i ≤ t3) into r + t h3 parts and embed them into the r check nodes of the sub-strip s i+1 and the t m+1 spaces of the extended sub-strip s h3 ; divide the k information data blocks of the sub-strip s i (t3 + 1 ≤ i < m) into r + t l3 parts and embed them into the r check nodes of the sub-strip s i+1 and the t m+1 spaces of the extended sub-strip s l3 ; Step C: Sub-strip s m The k information data blocks are divided equally into r+t l3 The copies are embedded into the r check nodes of sub-strip s1 and the extended sub-strip s as the Piggyback function. m+1 t l3 in a space.

4. A method for repairing a faulty node, wherein the method is used to repair the Piggybacking code with local repair characteristics obtained by the method for constructing the Piggybacking code with local repair characteristics according to any one of claims 1 to 3, and the specific operations are as follows: When a single node fails, the data block lost by the failed node is found in the local repair table, and then other non-lost data blocks in the column where the lost data block is located are downloaded and an XOR operation is performed to complete the repair of the data block; the other lost data blocks are repaired in the same way to complete the repair of the failed node.

Citation Information

Cited By

  • Single block repair cross-rack flow optimization method for piggyback coding

    CN120011132A