System MSR code generating and repairing method with minimum subpackage number

Abstract

The invention discloses a system MSR code generating and repairing method with the minimum subpackage number, and relates to the technical field of computers and distributed storage, and the method comprises the steps: defining the structure of a (n, k, d) system MSR code for a distributed storage system composed of n nodes; wherein the system MSR code is totally provided with n code blocks, the previous k code blocks serve as information nodes, the remaining last n-k code blocks serve as check nodes, all information is stored in the information nodes, the k information nodes are grouped, and at the moment, the subpackage number # imgabs0 # of the system MSR code calculates and generates (n, k, d) system MSR codes in a finite field based on a defined system MSR code structure. According to the method, the system MSR code is constructed by adopting a specific MSR construction mode, so that the system MSR code has a smaller subpackage number, the system MSR code with the minimum subpackage number is realized by combining fixed parameters with code calculation, and the corresponding check matrix is generated.

Description

Technical Field

[0001] The present invention relates to the technical field of computers and distributed storage, and in particular, to a method for generating and repairing a system MSR code with a minimum number of sub-packets. Background Art

[0002] The statements in this part only provide background technical information related to the present invention, and do not necessarily constitute prior art.

[0003] Distributed storage systems are widely used in modern cloud storage, Internet services, big data analysis and other scenarios, and play a huge role in global data centers. In particular, large cloud storage service providers such as Microsoft Azure, Amazon S3, and Google Cloud rely on efficient distributed storage systems to meet the growing data storage needs. In a distributed storage system, in order to improve the fault tolerance and reliability of the storage system, data is split and stored on multiple storage nodes. However, with the rapid growth of the amount of data, how to efficiently handle node failures and maintain the availability of data without wasting storage resources has become an important problem that needs to be solved in distributed storage systems.

[0004] Among existing erasure codes, Maximum Distance Separable (MDS) codes, such as Reed-Solomon (RS) codes, are widely used in distributed storage systems because they can handle the most node failures under the same storage efficiency, that is, all data can be recovered in any case of k nodes. The MSR code is a special type of MDS code that can repair a failed node with the minimum total network data transmission volume when a single node fails. Currently, a variety of studies on the construction of MSR codes have been proposed, such as the product matrix construction for the explicit construction of MSR codes; considering that this MSR code is limited to the case of low code rates, an explicit construction of high-rate MSR codes has also been proposed; for the case where the number of nodes d that can help in the repair during repair is d < n - 1 (n is the number of nodes in the system), the prior art gives a class of MSR code constructions with optimal access properties, and its number of sub-packets s = d - k + 1, d ∈ {k + 1, k + 2, k + 3}; and other structures of MSR codes, etc.

[0005] The size of the MSR code packet will affect the complexity of encoding and decoding and I / O (input / output) overhead: when the number of packets increases, the complexity of encoding and decoding will increase significantly. For example, when using scalar code, each node stores a scalar, and the complexity of each layer in the encoding and recovery process is O(rk), then the complexity of the l-th layer is O(rkl). When using an array code with a larger number of packets, each node stores a vector, and the complexity of the l-th layer encoding is O(rkl). 2 ); In addition, in the case of a large number of subpackets, MSR code will generate additional I / O overhead when repairing a single node. The increased I / O overhead is not suitable for actual storage systems. For example, in scalar code, when a node fails, all data needs to be transmitted through other nodes for recovery. Each node only needs to perform one I / O operation. When using array code, the recovery of the error node requires data transmission from more nodes to reduce the total amount of downloaded data. Since the data transmitted by each node is not necessarily continuous, this will lead to an increase in I / O overhead.

[0006] Therefore, the construction of MSR code must satisfy the following requirements: Under the condition that optimal access is not required, for all possible parameters (n, k, d), construct an MSR code with the smallest number of sub-packets as possible. s=d-k+1, but the number of subpackets of the MSR code is still large, and there is still room for optimization. Summary of the invention

[0007] In order to solve the above-mentioned deficiencies of the prior art, the present invention provides a system MSR code generation and repair method with a minimum number of subpackets, and designs a system MSR code with a minimum number of subpackets under the current given parameters (n, k, d). Based on the MSR code, a specific MSR construction method is used to construct the system MSR code so that it has a smaller number of subpackets. Then, fixed parameters are combined with code calculation to realize the system MSR code with the minimum number of subpackets and generate the corresponding check matrix.

[0008] In a first aspect, the present invention provides a method for generating a system MSR code with a minimum number of subpackets.

[0009] A method for generating a system MSR code with a minimum number of subpackets, comprising:

[0010] For a distributed storage system consisting of n nodes, the structure of the (n, k, f) system MSR code is defined; wherein the system MSR code has a total of n code blocks C = (C0, C1, ..., C n-1), the first k code blocks are used as information nodes, the remaining nk code blocks are used as check nodes, all information is stored in the information nodes, and the k information nodes are grouped. At this time, the number of packets of the system MSR code is

[0011] Based on the structure of the defined system MSR code, the (n, k, d) system MSR code is calculated and generated in a finite field as follows: using the default generated primitive irreducible polynomial f(x), any primitive element is selected in the finite field F2 / (f(x)), and a number of different elements are generated using the primitive element. Mapping is performed based on the generated elements to obtain the kernel matrix of each code block, and the corresponding check matrix is ​​determined according to the kernel matrix, thereby completing the generation of the system MSR code.

[0012] A further technical solution is to convert an element vector x [s] Mapping to a finite field F q The matrix of size st×s The mapping is represented as:

[0013]

[0014] Where b∈[s+1], t is a positive integer, is a block matrix of s×s, where each block is a column vector of length t; for b∈[s+1],i,j∈[s], The i-th row and j-th column of is represented as:

[0015]

[0016] In the above formula, Element x∈F q .

[0017] A further technical solution is to generate a (n=14, k=10, d=13) system MSR code, the generated system MSR code has a minimum number of subpackets s=d-k+1, d represents the number of nodes that help repair during repair;

[0018] The first k=10 code blocks are used as information nodes, the remaining last 4 code blocks are used as check nodes, all information is stored in the information nodes, and the 10 information nodes are grouped.

[0019] A further technical solution is to use the primitive irreducible polynomial f(x) generated by default, select any primitive element in the finite field F2 / (f(x)), and use the primitive element to generate several different elements, including:

[0020] Using the primitive irreducible polynomial f(x)=x 8 +x 4+x 3 +x 2 +1, generate the finite field GF(256) through F2 / (f(x)), expressed as:

[0021] GF(256)=F2 / (f(x))={a7x 7 +a6x 6 +...+a1x+a0|a i ∈{0,1}};

[0022] In the finite field GF(256), select a primitive element x and use it to generate several different elements. Set the element where each element λ i is a polynomial of degree at most 7.

[0023] A further technical solution is to determine the corresponding check matrix according to the kernel matrix, and define the check matrix as:

[0024]

[0025] In the above formula, I is the unit matrix, represents the Kronecker product, It represents the Kronecker product of each element block in matrix A and matrix B, a represents the number of divided groups, and b represents the bth information node in the group.

[0026] In a second aspect, the present invention provides a system MSR code repair method with a minimum number of subpackets.

[0027] A method for repairing a system MSR code with a minimum number of subpackets, comprising:

[0028] Repairing the information node of the (n=14, k=10, d=13) system MSR code generated by the method of the first aspect, and repairing the erroneous node according to different situations, including:

[0029] When an error occurs in any of the first b-1 nodes in group a, the set of helper nodes is |H|=13, then the error node is {R a,b C j :j∈H}repair;

[0030] When an error occurs in any of the last nodes in group a, the set of helper nodes is |H|=13, let:

[0031] H1=H∩(5a+[5]),a∈[2];

[0032] H2=H\(5a+[5],a∈[2]);

[0033] Then the last node C of group a 5a+4 By {R a,b C 5a+z :5a+z∈H1,b∈[4]} and {R a,b C i :i∈H2,b∈[4]}repair.

[0034] A further technical solution is that when any of the first four nodes C0 to C3 in the first group fails, each helping node C j Pass 4 discontinuous symbols to the error node for repair;

[0035] When any of the first four nodes C5 to C8 in the second group fails, each helper node C j Pass 4 consecutive symbols to the faulty node for repair.

[0036] In a third aspect, the present invention further provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-mentioned method for generating a system MSR code with a minimum number of subpackets when executing the executable instructions stored in the memory.

[0037] In a fourth aspect, the present invention further provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned method for generating a system MSR code with a minimum number of subpackets.

[0038] In a fifth aspect, the present invention also provides a computer program product, which includes executable instructions, and the executable instructions are stored in a computer-readable storage medium; wherein, when the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the above-mentioned method for generating a system MSR code with a minimum number of subpackets is implemented.

[0039] One or more of the above technical solutions have the following beneficial effects:

[0040] 1. The present invention provides a method for generating and repairing a system MSR code with a minimum number of subpackets. The method is designed with a minimum number of subpackets under the current given parameters (n, k, d). Based on the MSR code, a specific MSR construction method is used to construct the system MSR code to make it have a smaller number of subpackets. Then, fixed parameters are combined with code calculation to obtain the MSR code with 14 storage nodes, 10 information nodes, and 16 subpackets and its check matrix. In this way, the system MSR code with the minimum number of subpackets is realized, and the corresponding check matrix is ​​generated, which can effectively improve the storage efficiency and node fault recovery efficiency.

[0041] 2. The system MSR code proposed in the present invention optimizes storage and data repair. By designing the system MSR code, the system can achieve optimal repair in the event of failure of any given k nodes, and other check nodes no longer participate in the repair process. Since all information is stored in the information node, when downloading data, the user only needs to obtain it from the information node without involving the check node, which greatly simplifies the data recovery process, especially in a distributed storage system, reducing the user's dependence on the check node, so that only the rapid repair of the information node is needed, which can effectively reduce the complexity of the repair process and improve the efficiency of the system.

[0042] 3. The system MSR code proposed in the present invention realizes a smaller number of subpackets, reduces encoding complexity, and reduces I / O overhead. The present invention can realize a system MSR code with a smaller number of subpackets under the same parameters. A smaller number of subpackets means a smaller amount of data to be processed during the encoding process, thereby reducing the complexity of encoding and recovery at each layer, and by reducing the number of subpackets, the I / O overhead can be further reduced.

[0043] 4. The generation strategy of the system MSR code with a smaller number of subpackets adopted by the present invention not only significantly reduces the coding complexity and I / O overhead, but also optimizes the performance of the entire distributed storage system. In practical applications, this optimization can significantly improve the system repair speed, reduce resource consumption, and effectively improve the user experience.

[0044] 5. The system MSR code proposed in the present invention has the minimum repair bandwidth. When the MDS code is repairing data, it needs to read a large amount of data from multiple storage nodes to restore the lost data. Although the MDS code ensures that the data can be recovered from any k data blocks, because it needs to download k equal-sized data blocks to repair a single failed node, the repair bandwidth is very high. The MSR code uses the regeneration code technology, which only needs to download a small amount of data from other nodes, and restore the data of the failed node through linear combination. The minimum repair bandwidth is particularly important in large-scale distributed storage systems, which can significantly reduce the network burden, that is, by reducing the amount of transmitted data to improve the repair efficiency, reduce the system's requirements for network bandwidth; the efficiency of repairing a single failed node is high. When repairing a single node failure, the MDS code needs to read the complete data block from the other k nodes and restore the lost data through decoding operations. This method causes the recovery process to not only consume more bandwidth, but also increase the node load. The MSR code is optimized for the repair of a single failed node, and only needs to read the linear combination of data fragments from some nodes, without downloading all the data. The repair process is more efficient than that of MDS code, which not only reduces the amount of transmitted data, but also reduces the recovery time, thereby reducing the load pressure of the system during data recovery; optimize storage redundancy. MDS code has high storage efficiency and can achieve less redundancy (theoretically close to the optimal), that is, the total number of stored data blocks is n, and any k data blocks can restore all data. Its storage overhead is fixed and redundancy is less. MSR code can achieve the minimum storage overhead like MDS code, and the storage efficiency is also very high, but MSR code takes into account the optimization of repair efficiency and bandwidth at the same time, making the repair process more efficient under the same redundancy; optimize multi-node repair. When errors occur in multiple nodes of MDS code, a large amount of data needs to be downloaded to restore these nodes. Its repair bandwidth demand increases linearly with the increase of the number of failed nodes, resulting in an exponential increase in the repair cost. MSR code can repair the error node by combining the remaining nodes, thereby reducing the repair bandwidth. Therefore, when multiple nodes fail at the same time, the repair efficiency is higher than that of MDS code.

[0045] 6. The system MSR code proposed in the present invention is more suitable for network environments. The high bandwidth requirement of the MDS code limits its use in environments with limited network bandwidth. For scenarios with poor network conditions or high bandwidth costs, the repair cost of the MDS code is very high, while the MSR code minimizes the amount of data transmitted during the repair process, reduces the bandwidth pressure during the repair process, and improves the overall operating efficiency of the system. Therefore, it performs better in environments with limited bandwidth and is suitable for bandwidth-sensitive applications, such as data transmission across data centers and edge computing scenarios.

[0046] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0048] Figure 1 A flowchart of a method for generating a system MSR code with a minimum number of subpackets according to an embodiment of the present invention;

[0049] Figure 2 Schematic diagram of polynomial form, binary form, and hexadecimal form of various elements generated in an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of calculating the H4 matrix value in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of repairing when an error occurs in any node among the first four nodes C0 to C3 of the first group in an embodiment of the present invention;

[0052] Figure 5 It is a schematic diagram of repairing when an error occurs in any node among the first four nodes C5 to C8 of the second group in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] It should be noted that the following detailed descriptions are exemplary only, are intended to describe specific embodiments, are intended to provide further explanation of the present invention, and are not intended to limit exemplary embodiments according to the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art to which the present invention belongs. In addition, it should also be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0054] Embodiment 1

[0055] This embodiment provides a method for generating a system MSR code with a minimum number of subpackets, such as Figure 1 As shown, specifically including:

[0056] First, the minimum number of sub-packets designed under the current given parameters (n, k, d) Based on the MSR code, a specific MSR construction method is used to construct the system MSR code, so that it has a smaller number of subpackets, specifically:

[0057] For a distributed storage system consisting of n nodes, the structure of the (n, k, d) system MSR code is defined; wherein the system MSR code has a total of n code blocks C = (C0, C1, ..., C n-1 ), the first k code blocks are used as information nodes, the remaining nk code blocks are used as check nodes, all information is stored in the information nodes, and the k information nodes are grouped. At this time, the number of packets of the system MSR code is

[0058] Secondly, based on the structure of the system MSR code defined above, the (n, k, d) system MSR code is calculated and generated in a finite field by using fixed parameters combined with code calculation, so as to realize the system MSR code with the minimum number of subpackets and generate the corresponding check matrix, which is specifically:

[0059] Using the default generated primitive irreducible polynomial f(x), select any primitive element in the finite field F2 / (f(x)), use the primitive element to generate several different elements, map based on the generated elements, obtain the kernel matrix of each code block, and determine the corresponding check matrix based on the kernel matrix, so as to complete the generation of the system MSR code.

[0060] The following content introduces the system MSR code generation method with the minimum number of subpackets proposed in this embodiment in more detail.

[0061] Before generating the system MSR code, the following definitions are given:

[0062] (1) For an element x∈F q and a positive integer t, let:

[0063]

[0064] (2) For any matrix A and a block matrix B of size m×n, where:

[0065]

[0066] Then define:

[0067]

[0068] in, is the Kronecker product, This means that the Kronecker product of each element block in matrix A and matrix B is performed.

[0069] (3) For b∈[s+1] and a positive integer t, define the following mapping:

[0070]

[0071] That is, a vector x [s] Mapping to a finite field F q The matrix of size st×s Can is viewed as a block matrix of s×s, where each block is a column vector of length t. Then, for b∈[s+1],i,j∈[s], The i-th row and j-th column of can be expressed as:

[0072]

[0073] Furthermore, for ease of understanding, s=3 is used as an example. Based on the above formula, when b∈[4],i,j∈[3], we have:

[0074]

[0075] (4) For any non-negative integers a and b, define R a,b for:

[0076]

[0077] In the above formula, e b represents a vector of length s, with the bth bit being 1 and the rest being 0. m represents the identity matrix of size m×m, where m is l / s a+1 、s a , l and s are both non-negative integers, l represents the number of sub-packets.

[0078] Based on the above definition, a system MSR code with the minimum number of subpackets is constructed. Specifically, suppose a distributed storage system consisting of n nodes is constructed. The system MSR code performs distributed storage of the data or files that need to be stored. Among them, the system MSR code has a total of n code blocks C = (C0, C1, ..., C n-1 ), the first k code blocks are used as information nodes, the remaining nk code blocks are used as check nodes, all information is stored in the information nodes, and the k information nodes are grouped. At this time, the number of packets of the system MSR code is s=d-k+1, d represents the number of nodes that help with repair, and l represents the number of subpackets.

[0079] In this embodiment, a (n=14, k=10, d=13) system MSR code is generated. For this system MSR code, the first 10 code blocks are information nodes, which are divided into 2 groups, each with 5 code blocks. All data information is stored only through these 10 code blocks; the remaining 4 code blocks are used as check nodes, and the check nodes are not grouped; at this time, s=d-k+1=4, the number of subpackets is Compared with the existing optimal MSR code (n=14, k=10), the number of sub-packets is Under the same parameters, the system MSR code proposed in this embodiment only considers the optimal repair of 10 information nodes, and its subpacket number is This number of packets can be further reduced, thereby effectively reducing the complexity of encoding and decoding and I / O overhead.

[0080] Furthermore, for the convenience of representation, a∈[2],b∈[5] is used to represent the bth node of the ath group, that is, the first 10 nodes can be represented as (C 5a+b :a∈[2],b∈[5]). On this basis, the generation process of the MSR code of the (n=14,k=10,d=13) system is introduced.

[0081] First, use the primitive irreducible polynomial f(x) generated by the system by default, generate a finite field GF(256) through F2 / (f(x)), and arbitrarily select a primitive element x on the finite field GF(256). Use the primitive element x to generate several different elements, and you can set the element λ i =x i , i = 0, 1, ..., 43. Specifically, this embodiment uses the mathematical software system sagemath, and the primitive irreducible polynomial f(x) generated by default in the system is: f(x) = x 8 +x 4 +x 3 +x 2 +1.

[0082] The finite field GF(256) is generated by F2 / (f(x)), which can be expressed as:

[0083] GF(256)=F2 / (f(x))={a7x 7 +a6x 6 +...+a1x+a0|a i ∈{0,1}}.

[0084] Select a primitive element x in the finite field GF(256) to generate several different elements, where each element λ i is a polynomial of degree at most 7, and the resulting λ0,λ1,...,λ 43 The polynomial form, binary and hexadecimal representations are as follows Figure 2 shown.

[0085] Secondly, based on the generated elements, combined with the mapping defined above Mapping is performed to obtain the kernel matrix of each code block. Specifically, let According to the above definition mapping Get the kernel matrix a∈[2],b∈[5],t=r, that is:

[0086]

[0087]

[0088] Finally, the corresponding check matrix is ​​determined according to the kernel matrix to complete the generation of the system MSR code. Specifically, the check matrix is ​​defined according to the kernel matrix as:

[0089]

[0090] Taking H4 as an example, according to the above definition, H4 can be obtained as:

[0091]

[0092] According to the definition of the check matrix above, the specific value of each matrix is ​​calculated. Taking the H4 matrix as an example, the mathematical software system sagemath is used to calculate the H4 matrix value, as follows Figure 3 shown.

[0093] Embodiment 2

[0094] Based on the above-mentioned system MSR code generation method, this embodiment further provides a system MSR code repair method with a minimum number of subpackets, including:

[0095] The (n=14, k=10, d=13) system MSR code generated in the first embodiment is used to repair information nodes. The error nodes are repaired according to different situations according to the different nodes where errors occur.

[0096] First, let’s define node repair. Assume that one of the nodes has an error, and the error node is C. i , i∈

[10] , the auxiliary node is C j , j∈

[14] \{i}, since the node repair of MSR code needs to be downloaded from each auxiliary node symbols, so D j→i It is designed as a 4×16 matrix, which means that when an error occurs in the i-th node, the j-th node chooses which part of the data to download to the error node C i , that is, from the auxiliary node C j The transmitted data can be expressed as D j→i C j , and then the (n-1)β=52 symbols downloaded from all auxiliary nodes can be expressed as:

[0097]

[0098] Then, the error node C i After obtaining the transmitted data, the repair matrix R is used according to these data. i Repairing the error block to restore the original data can be expressed as:

[0099]

[0100] Through the above method, the error block can be repaired.

[0101] Secondly, according to the different nodes where the errors occur, the error nodes are repaired according to different situations.

[0102] (1) Case 1

[0103] When an error occurs in any of the first b-1 nodes in group a, the node where the error occurs is C. 5a+b ,a∈[2],b∈[4], the set of helping nodes is |H|=13, then the error node can be {R a,b C j :j∈H} repair.

[0104] Furthermore, according to the formula We can get:

[0105]

[0106] Among them, e i represents a row vector of length 4, with the i-th bit being 1 and the rest being 0, i = 0, 1, 2, 3; R a,b It indicates that the b-th node of the a-th group selects the data to be transmitted to the wrong node.

[0107] When C0 fails, according to the above R 0,0 It can be seen that each helping node C j Pass the 1st, 5th, 9th, and 13th symbols to C0, that is, C j (0),C j (4),C j (8),C j (12). The rest is similar, as follows Figure 4 As shown, when any of the first four nodes C0 to C3 in the first group fails, each helper node C j Four discontinuous symbols are passed to the error node for repair, which requires four I / O operations in total. Therefore, C0, C1, C2, and C3 store relatively important data.

[0108] like Figure 5As shown, when any of the first four nodes C5 to C8 in the second group fails, each helper node C j Passing 4 consecutive symbols to the faulty node for repair requires only 1 I / O operation, so C5, C6, C7, and C8 can save the most important data.

[0109] (2) Second Case

[0110] When an error occurs in any of the last nodes in group a (i.e., C4, C9), the set of helper nodes is |H|=13, let:

[0111] H1=H∩(5a+[5]),a∈[2];

[0112] H2=H\(5a+[5],a∈[2]);

[0113] Then the last node C of group a 5a+4 Can be {R a,b C 5a+z :5a+z∈H1,b∈[4]} and {R a,b C i :i∈H2,b∈[4]} repair, that is, the helper nodes in the same group pass data to the error node {R a,b C i :i∈H2},b∈[4], different groups of nodes transmit data to the error node {R a,b C 5a+Z :5a+z∈H1,b∈[4]}.

[0114] Specifically, when an error occurs in C4, the data transmitted by each node to C4 is:

[0115]

[0116]

[0117] When an error occurs in C9, the data transmitted by each node to C9 is:

[0118]

[0119] Through the above-mentioned method, this embodiment realizes the construction and generation of a systematic MSR code (Systematic MSRCode) with a minimum number of subpackets. Compared with the existing general MSR code (General MSR Code), the systematic MSR code proposed in this embodiment further optimizes storage and data repair. That is, the systematic MSR code proposed in this embodiment can store data including not only redundant coding parts but also directly stored original data parts. The systematic MSR code can achieve optimal repair in the case of any k nodes, while other check nodes do not consider optimal repair. Since all information is stored in the information node, users only need to obtain data from the information node when downloading data, which is independent of the check node. Therefore, in the actual distributed storage system, only the rapid repair of the information node needs to be paid attention to. Different from the existing general MSR code that considers the optimal repair of all nodes, the systematic MSR code proposed in this embodiment only focuses on the optimal repair of 10 information nodes and ignores the repair problem of 4 check nodes, thereby achieving the optimization of storage and data repair.

[0120] More importantly, compared with the existing (n=14, k=10) MSR code structure, the number of sub-packets is The system MSR code proposed in this embodiment can achieve a smaller number of subpackets under the same parameters: Since this embodiment only considers the optimal repair of 10 information nodes, the number of subpackets is This embodiment can reduce the encoding complexity and effectively reduce I / O overhead by reducing the number of subpackets, further optimizing the overall performance. In addition, the system MSR code proposed in this embodiment only needs to download a small amount of data from other nodes during repair, and restore the data of the failed node through linear combination. It has the minimum repair bandwidth and is more efficient in repairing a single failed node, which can optimize storage redundancy, optimize multi-node repair, etc.

[0121] In addition, the use of the system MSR code proposed above can improve the scalability of the system: in large-scale storage systems, MSR codes can handle more node failure scenarios while ensuring data reliability, which is very important for systems with high scalability requirements; energy efficiency optimization: since MSR codes reduce the bandwidth required for data transmission and repair, they can reduce the system's energy consumption during data recovery, especially in large-scale data centers; more flexible design and implementation: MSR codes can achieve flexible trade-offs between storage overhead, repair bandwidth and repair complexity through different parameter designs to adapt to the needs of different application scenarios.

[0122] Embodiment 3

[0123] This embodiment provides an electronic device, including: a memory, used to store executable instructions; and a processor, used to implement the above method provided in this embodiment when executing the executable instructions stored in the memory.

[0124] Embodiment 4

[0125] This embodiment also provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by a processor, the processor will be caused to execute the above method provided in this embodiment.

[0126] Embodiment 5

[0127] This embodiment provides a computer program product, which includes an executable instruction, which is a computer instruction; the executable instruction is stored in a computer-readable storage medium. When a processor of an electronic device reads the executable instruction from the computer-readable storage medium and the processor executes the executable instruction, the electronic device executes the above method provided in this embodiment.

[0128] The steps involved in the above embodiments 2 to 5 correspond to the method embodiment 1. For the specific implementation, please refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0129] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0130] The above description is only a preferred embodiment of the present invention. Although the specific implementation mode of the present invention is described in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that on the basis of the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. A method for generating a system MSR code with a minimum number of subpackets, characterized in that: include: For a distributed storage system consisting of n nodes, the structure of the (n, k, d) system MSR code is defined; wherein the system MSR code has a total of n code blocks C = (C0, C1, ..., C n-1 ), the first k code blocks are used as information nodes, the remaining nk code blocks are used as check nodes, all information is stored in the information nodes, and the k information nodes are grouped. At this time, the number of packets of the system MSR code is Based on the structure of the defined system MSR code, the (n, k, d) system MSR code is calculated and generated in a finite field as follows: using the default generated primitive irreducible polynomial f(x), any primitive element is selected in the finite field F2 / (f(x)0, and a number of different elements are generated using the primitive element. Mapping is performed based on the generated elements to obtain the kernel matrix of each code block, and the corresponding check matrix is ​​determined according to the kernel matrix, thereby completing the generation of the system MSR code.

2. A method for generating a system MSR code with a minimum number of subpackets as claimed in claim 1, characterized in that: Transform an element vector x [s] Mapping to a finite field F q The matrix of size st×s The mapping is represented as: Where b∈[s+1], t is a positive integer, is a block matrix of s×s, where each block is a column vector of length t; for b∈[s+1],i,j∈[s], The i-th row and j-th column of is represented as: In the above formula, Element x∈F q .

3. A method for generating a system MSR code with a minimum number of subpackets as claimed in claim 1, characterized in that: Generate (n=14, k=10, d=13) system MSR code, the generated system MSR code has the minimum number of subpackets s=d-k+1, d represents the number of nodes that help repair during repair; The first k=10 code blocks are used as information nodes, the remaining last 4 code blocks are used as check nodes, all information is stored in the information nodes, and the 10 information nodes are grouped.

4. A method for generating a system MSR code with a minimum number of subpackets as claimed in claim 3, characterized in that: Using the default generated primitive irreducible polynomial f(x), select any primitive element in the finite field F2 / (f(x)0, and use the primitive element to generate several different elements, including: Using the primitive irreducible polynomial f(x)=x 8 +x 4 +x 3 +x 2 +1, generate the finite field GF(256) through F2 / (f(x)), expressed as: GF(256)=F2 / (f(x))={a7x 7 +a6x 6 +...+a1x+a0|a i ∈{0,1}}; In the finite field GF(256), select a primitive element x, use it to generate several different elements, and set the element λ i =x i , where each element λ i is a polynomial of degree at most 7.

5. A method for generating a system MSR code with a minimum number of subpackets as claimed in claim 3, characterized in that: Determine the corresponding check matrix according to the kernel matrix, and define the check matrix as: In the above formula, I is the unit matrix, represents the Kronecker product, It represents the Kronecker product of each element block in matrix A and matrix B, a represents the number of divided groups, and b represents the bth information node in the group.

6. A method for repairing a system MSR code with a minimum number of subpackets, characterized in that: include: Repairing the information node of the (n=14, k=10, d=13) system MSR code generated by the method according to any one of claims 1 to 5, and repairing the erroneous node according to different situations, including: When an error occurs in any of the first b-1 nodes in group a, the set of helper nodes is The error node is {R a,b C j :j∈H}repair; When an error occurs in any of the last nodes in group a, the set of helper nodes is set up: H1=H∩(5a+[5]),a∈[2]; H2=H\(5a+[5],a∈[2]); Then the last node C of group a 5a+4 By {R a,b C 5a+z :5a+z∈H1,b∈[4]} and {R a,b C i :i∈H2,b∈[4]}repair.

7. A method for repairing a system MSR code with a minimum number of subpackets as claimed in claim 6, characterized in that: include: When any of the first four nodes C0 to C3 in the first group fails, each helper node C j Pass 4 discontinuous symbols to the error node for repair; When any of the first four nodes C5 to C8 in the second group fails, each helper node C j Pass 4 consecutive symbols to the faulty node for repair.

8. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the system MSR code generation method with the minimum number of subpackets as described in any one of claims 1 to 5 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause the processor to execute the executable instructions to implement the system MSR code generation method with the minimum number of subpackets as described in any one of claims 1-5.

10. A computer program product, characterized in that The computer program product includes executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the method for generating a system MSR code with a minimum number of subpackets as described in any one of claims 1 to 5 is implemented.

Images