A method for generating and repairing MSR codes in a system with a minimum number of packets.

Abstract

This invention discloses a method for generating and repairing a system MSR code with the minimum number of packets, relating to the fields of computer and distributed storage technology. The method includes: for a distributed storage system consisting of n nodes, defining the structure of an (n,k,d) system MSR code; wherein the system MSR code has n code blocks, the first k code blocks are used as information nodes, and the remaining n-k code blocks are used as check nodes. All information is stored in the information nodes, and the k information nodes are grouped. The number of packets in the system MSR code is then calculated based on the defined system MSR code structure, generating the (n,k,d) system MSR code in a finite field. This invention employs a specific MSR construction method to construct the system MSR code, enabling it to have a smaller number of packets, and utilizes fixed parameters combined with code calculation to achieve a system MSR code with the minimum number of packets, and generates the corresponding check matrix.

Description

Technical Field

[0001] The present invention relates to the technical fields of computers and distributed storage, and particularly to a method for generating and repairing a system MSR code with a minimum number of sub-packets. Background Art

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Distributed storage systems are widely used in modern cloud storage, Internet services, big data analysis and other scenarios, and play a huge role in global data centers. Especially large cloud storage service providers such as Microsoft Azure, Amazon S3, and Google Cloud rely on efficient distributed storage systems to meet the growing data storage needs. In a distributed storage system, in order to improve the fault tolerance and reliability of the storage system, data is split and stored on multiple storage nodes. However, with the rapid growth of the amount of data, how to efficiently handle node failures and maintain the availability of data without wasting storage resources has become an important issue that needs to be solved in distributed storage systems.

[0004] Among existing erasure codes, Maximum Distance Separable (MDS) codes, such as Reed-Solomon (RS) codes, are widely used in distributed storage systems because they can handle the most node failures under the same storage efficiency, that is, all data can be recovered in the case of any k nodes. The MSR code is a special type of MDS code that can repair a failed node with the minimum total network data transmission volume when a single node fails. Currently, many studies on the construction of MSR codes have been proposed, such as the product matrix construction for the explicit construction of MSR codes; considering that this MSR code is limited to the case of low code rates, an explicit construction of high-rate MSR codes has also been proposed; for the case where the number of nodes d that can help with repair during repair is less than n - 1 (n is the number of nodes in the system), the prior art gives a class of MSR code constructions with optimal access properties, and the number of sub-packets s = d - k + 1, d ∈ {k + 1, k + 2, k + 3}; and other structures of MSR codes, etc.

[0005] The number of packets in an MSR code affects the complexity of encoding and decoding, as well as I / O (input / output) overhead. As the number of packets increases, the complexity of encoding and decoding increases significantly. For example, when using scalar code, each node stores a scalar, and the complexity of each layer in the encoding and recovery process is O(rk), then the complexity of layer l is O(rkl). However, when using array code with a larger number of packets, each node stores a vector, and the complexity of layer l encoding is O(rkl). 2 Furthermore, with a large number of packets, MSR codes will generate additional I / O overhead during single-node repair. Increased I / O overhead makes them unsuitable for practical storage systems. For example, in scalar codes, when a node fails, all data needs to be transferred through other nodes for recovery, and each node only needs to perform one I / O operation. However, when using array codes, the recovery of the faulty node requires data transfer from more nodes to reduce the total amount of data downloaded. Since the data transferred by each node is not necessarily continuous, this leads to increased I / O overhead.

[0006] Therefore, the construction of MSR codes must satisfy the following condition: without requiring optimal access, for all possible parameters (n,k,d), construct an MSR code with the smallest possible number of packets. Currently, the MSR code with the minimum number of packets designed under given parameters (n,k,d) has the following packet count... s = d - k + 1, but the number of packets in this MSR code is still relatively large, and there is still room for optimization. Summary of the Invention

[0007] To address the shortcomings of the existing technology, this invention provides a method for generating and repairing system MSR codes with the minimum number of packets, which designs a method with the minimum number of packets under given parameters (n,k,d). Based on the existing MSR code, a specific MSR construction method is used to construct the system MSR code, which has a smaller number of packets. Then, by using fixed parameters in combination with code calculation, a system MSR code with the minimum number of packets is realized, and a corresponding check matrix is ​​generated.

[0008] In a first aspect, the present invention provides a method for generating system MSR codes with the minimum number of packets.

[0009] A method for generating MSR codes in a system with a minimum number of packets includes:

[0010] For a distributed storage system consisting of n nodes, define the structure of the (n,k,f) system MSR code; where the system MSR code has n code blocks C = (C0,C1,…,C…). n-1The system uses the first k code blocks as information nodes and the remaining nk code blocks as check nodes. All information is stored in the information nodes, and the k information nodes are grouped together. The number of packets in the system's MSR code is as follows:

[0011] Based on the defined structure of the system MSR code, the (n,k,d) system MSR code is generated in a finite field as follows: using the default generated primitive irreducible polynomial f(x), a primitive element is randomly selected in the finite field F2 / (f(x)), and several different elements are generated using this primitive element. Based on the generated elements, a mapping is performed to obtain the kernel matrix of each code block, and the corresponding parity check matrix is ​​determined according to the kernel matrix, thereby completing the generation of the system MSR code.

[0012] A further technical solution is to use an element vector x [s] Mapped to a finite field F q A matrix of size st×s The mapping is represented as follows:

[0013]

[0014] Where b∈[s+1], t is a positive integer. Let be an s×s block matrix, where each block is a column vector of length t; for b∈[s+1], i,j∈[s], The i-th row and j-th column is represented as:

[0015]

[0016] In the above formula, element x∈F q .

[0017] A further technical solution involves generating (n=14, k=10, d=13) system MSR codes, which possess the minimum number of packets required. s = d - k + 1, where d represents the number of nodes that help with the repair process;

[0018] Previously, k=10 code blocks were used as information nodes, and the remaining 4 code blocks were used as check nodes. All information was stored in the information nodes, and the 10 information nodes were grouped together.

[0019] A further technical solution utilizes the default-generated primitive irreducible polynomial f(x), and selects any primitive element in the finite field F2 / (f(x)). Using this primitive element, several different elements are generated, including:

[0020] Using the primitive irreducible polynomial f(x) = x 8 +x 4+x 3 +x 2 +1, generating the finite field GF(256) through F2 / (f(x)), expressed as:

[0021] GF(256)=F2 / (f(x))={a7x 7 +a6x 6 +...+a1x+a0|a i ∈{0,1}};

[0022] In the finite field GF(256), choose any primitive element x, and use this primitive element to generate several different elements, defining the elements... Each element λ i It is a polynomial with a maximum degree of 7.

[0023] A further technical solution involves determining the corresponding parity check matrix based on the kernel matrix, and defining the parity check matrix as follows:

[0024]

[0025] In the above formula, I is the identity matrix. Indicates the Kronecker product. This represents the Kronecker product of the element blocks in matrix A and matrix B, where 'a' represents the number of groups and 'b' represents the b-th information node within the group.

[0026] Secondly, the present invention provides a system MSR code repair method with the minimum number of packets.

[0027] A system MSR code repair method with minimal packet number includes:

[0028] The information nodes of the (n=14, k=10, d=13) system MSR code generated using the method described in the first aspect are repaired. Depending on the specific node where the error occurred, the erroneous nodes are repaired in different ways, including:

[0029] When any of the first b-1 nodes in group a encounters an error, the set of helper nodes is: If |H| = 13, then the faulty node is {R} a,b C j Repair:j∈H}

[0030] When any node in the last node of group a encounters an error, the set of helper nodes is: |H|=13, let:

[0031] H1=H∩(5a+[5]),a∈[2];

[0032] H2=H\(5a+[5],a∈[2]);

[0033] Then the last node C of group a 5a+4 by {R a,b C 5a+z :5a+z∈H1,b∈[4]} and {R a,b C i :i∈H2,b∈[4]}Repair.

[0034] A further technical solution is that when any of the first four nodes C0 to C3 in the first group encounters an error, each helper node C... j To repair the erroneous node, pass four discontinuous symbols.

[0035] When any of the first four nodes C5 to C8 in the second group encounters an error, each helper node C j Send four consecutive symbols to the faulty node to repair it.

[0036] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-described method for generating a system MSR code with a minimum number of packets when executing the executable instructions stored in the memory.

[0037] Fourthly, the present invention also provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-described method for generating a system MSR code with a minimum number of packets.

[0038] Fifthly, the present invention also provides a computer program product comprising executable instructions stored in a computer-readable storage medium; wherein, when the processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, it implements the above-described method for generating a system MSR code with a minimum number of packets.

[0039] The above one or more technical solutions have the following beneficial effects:

[0040] 1. This invention provides a method for generating and repairing system MSR codes with the minimum number of packets, which designs a method with the minimum number of packets under given parameters (n,k,d). Based on the existing MSR code, a specific MSR construction method is used to construct the system MSR code, which has a smaller number of packets. Then, by using fixed parameters and code calculation, an MSR code with 14 storage nodes, 10 information nodes, and 16 packets and its parity check matrix are obtained. This achieves a system MSR code with the minimum number of packets and generates a corresponding parity check matrix, which can effectively improve storage efficiency and node fault recovery efficiency.

[0041] 2. The system MSR code proposed in this invention optimizes storage and data recovery. By designing the system MSR code, the system can achieve optimal recovery even if any k nodes fail, while other verification nodes no longer participate in the recovery process. Since all information is stored in the information nodes, users only need to retrieve data from the information nodes when downloading data, without involving the verification nodes. This greatly simplifies the data recovery process, especially in distributed storage systems, reducing users' dependence on verification nodes. Users can then focus on the rapid recovery of the information nodes, effectively reducing the complexity of the recovery process and improving system efficiency.

[0042] 3. The system MSR code proposed in this invention achieves a smaller number of packets, reduces coding complexity, and decreases I / O overhead. This invention can achieve a system MSR code with a smaller number of packets under the same parameters. A smaller number of packets means less data needs to be processed during the encoding process, thereby reducing the complexity of each layer of encoding and recovery. Furthermore, by reducing the number of packets, I / O overhead can be further reduced.

[0043] 4. The system MSR code generation strategy with a smaller number of packets adopted in this invention not only significantly reduces encoding complexity and I / O overhead, but also optimizes the performance of the entire distributed storage system. In practical applications, this optimization can significantly improve the system's repair speed, reduce resource consumption, and effectively enhance the user experience.

[0044] 5. The MSR code proposed in this invention has the lowest possible repair bandwidth. When repairing data, MDS codes require reading a large amount of data from multiple storage nodes. Although MDS codes guarantee data recovery from any k data blocks, the repair bandwidth is very high because it requires downloading k equally sized data blocks to repair a single failed node. In contrast, MSR codes use regenerable code technology, requiring only a small amount of data to be downloaded from other nodes. Data from the failed node is recovered through linear combination. This lowest repair bandwidth is particularly important in large-scale distributed storage systems, significantly reducing network load by improving repair efficiency and lowering the system's network bandwidth requirements. It also offers high efficiency in repairing a single failed node. When repairing a single node failure, MDS codes require reading complete data blocks from k other nodes and recovering the lost data through decoding. This method not only consumes a lot of bandwidth but also increases node load. MSR codes, however, are optimized for repairing single failed nodes, requiring only the reading of linearly combined data fragments from a subset of nodes, eliminating the need to download all data. The repair process is more efficient than MDS codes, reducing both the amount of data transmitted and the recovery time, thus alleviating the system's load during data recovery; it also optimizes storage redundancy. While MDS codes have high storage efficiency and can achieve minimal redundancy (theoretically close to optimal), meaning that with a total of n data blocks, any k data blocks can recover all data, and its storage overhead is fixed with minimal redundancy. MSR codes, like MDS codes, achieve minimal storage overhead and very high storage efficiency, but they also optimize both repair efficiency and bandwidth, making the repair process more efficient under the same redundancy conditions; furthermore, it optimizes multi-node repair. When multiple nodes fail, MDS codes require downloading a large amount of data to recover these nodes, and the repair bandwidth requirement increases linearly with the number of failed nodes, leading to a significant increase in repair costs. MSR codes, however, can repair erroneous nodes by combining the remaining nodes, thus reducing repair bandwidth. Therefore, when multiple nodes fail simultaneously, the repair efficiency is higher than that of MDS codes.

[0045] 6. The MSR code proposed in this invention is more suitable for network environments. The high bandwidth requirement of MDS codes limits their use in environments with limited network bandwidth. For scenarios with poor network conditions or high bandwidth costs, the repair cost of MDS codes is very high. In contrast, MSR codes minimize the amount of data transmitted during the repair process, reduce bandwidth pressure during the repair process, and improve the overall operating efficiency of the system. Therefore, they perform better in environments with limited bandwidth and are suitable for bandwidth-sensitive applications, such as cross-data center data transmission and edge computing scenarios.

[0046] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0047] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0048] Figure 1 This is a flowchart of the system MSR code generation method with minimum number of packets according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram showing the polynomial, binary, and hexadecimal forms of various elements generated in the embodiments of the present invention.

[0050] Figure 3 This is a schematic diagram illustrating the calculated H4 matrix values ​​in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram illustrating the repair process when any of the first four nodes C0 to C3 in the first group of the present invention encounters an error in an embodiment of the present invention.

[0052] Figure 5 This is a schematic diagram illustrating the repair process when any of the first four nodes C5 to C8 in the second group malfunctions in an embodiment of the present invention. Detailed Implementation

[0053] It should be noted that the following detailed descriptions are exemplary and are intended only to describe specific embodiments and to provide further explanation of the invention, and are not intended to limit the scope of exemplary embodiments of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0054] Example 1

[0055] This embodiment provides a system MSR code generation method with the minimum number of packets, such as... Figure 1 As shown, it specifically includes:

[0056] First, design a scheme with the minimum number of packets given the current parameters (n,k,d). Based on the existing MSR code, a specific MSR construction method is used to construct the system MSR code, which results in a smaller number of packets. Specifically:

[0057] For a distributed storage system consisting of n nodes, define the structure of the (n,k,d) system MSR code; where the system MSR code has n code blocks C = (C0,C1,…,C…). n-1 The system uses the first k code blocks as information nodes and the remaining nk code blocks as check nodes. All information is stored in the information nodes, and the k information nodes are grouped together. The number of packets in the system's MSR code is as follows:

[0058] Secondly, based on the structure of the system MSR code defined above, using fixed parameters combined with code calculation, an (n,k,d) system MSR code is generated in a finite field, thereby realizing a system MSR code with the minimum number of packets and generating the corresponding parity check matrix, specifically:

[0059] Using the default generated primitive irreducible polynomial f(x), a primitive element is randomly selected in the finite field F2 / (f(x)). Several different elements are generated using this primitive element. Based on the generated elements, a mapping is performed to obtain the kernel matrix of each code block. The corresponding parity check matrix is ​​determined according to the kernel matrix, thereby completing the generation of the system MSR code.

[0060] The following content provides a more detailed description of the system MSR code generation method with the minimum number of packets proposed in this embodiment.

[0061] Before generating the system MSR code, the following definitions are given:

[0062] (1) For element x∈F q Let t be a positive integer.

[0063]

[0064] (2) For any matrix A and a block matrix B of size m×n, where:

[0065]

[0066] Then define:

[0067]

[0068] in, It is the Kronecker product. This indicates that the Kronecker product is performed between the blocks of elements in matrix A and matrix B.

[0069] (3) For b∈[s+1] and positive integer t, the following mapping is defined:

[0070]

[0071] That is, to give a vector x [s] Mapped to a finite field F q A matrix of size st×s Can Consider it as an s×s block matrix, where each block is a column vector of length t. Then, for b∈[s+1], i,j∈[s], The i-th row and j-th column can be represented as:

[0072]

[0073] Furthermore, for ease of understanding, let's take s=3 as an example. Based on the above formula, when b∈[4], i,j∈[3], then we have:

[0074]

[0075] (4) For any non-negative integers a, b, define R a,b for:

[0076]

[0077] In the above formula, e b I represents a vector of length s, where the b-th bit is 1 and all other bits are 0. m Let m represent an identity matrix of size m × m, where m is l / s. a+1 s a Both l and s are non-negative integers, where l represents the number of packages.

[0078] Based on the above definition, a system MSR code with the minimum number of packets is constructed. Specifically, assuming a distributed storage system consisting of n nodes, the following is constructed: The system MSR code is used for distributed storage of data or files. The system MSR code consists of n code blocks C = (C0, C1, ..., C...). n-1 The system uses the first k code blocks as information nodes and the remaining nk code blocks as check nodes. All information is stored in the information nodes, and the k information nodes are grouped together. The number of packets in the system's MSR code is as follows: s = d - k + 1, where d represents the number of nodes that help with the repair, and l is the size of the sub-packets.

[0079] In this embodiment, a system MSR code (n=14, k=10, d=13) is generated. For this system MSR code, the first 10 code blocks are information nodes, divided into two groups of 5 blocks each. All data information is stored only through these 10 code blocks. The remaining 4 code blocks are used as check nodes, and these check nodes are not grouped. At this point, s = d - k + 1 = 4, and the number of packets... Compared to the existing optimal MSR code (n=14, k=10), its packet number is Under the same parameters, the MSR code proposed in this embodiment only considers the optimal repair of 10 information nodes, and its packet number is... This number of packets can be further reduced, thereby effectively reducing the complexity of encoding and decoding as well as I / O overhead.

[0080] Furthermore, for ease of representation, we use a∈[2], b∈[5] to represent the b-th node of the a-th group, that is, the first 10 nodes can be represented as (C 5a+b :a∈[2],b∈[5]). Based on this, the generation process of the MSR code of the (n=14,k=10,d=13) system is introduced.

[0081] First, using the system-generated primitive irreducible polynomial f(x), a finite field GF(256) is generated through F2 / (f(x)). A primitive element x is arbitrarily selected over this finite field GF(256), and several different elements are generated using this primitive element x. The element λ can be set. i =x i Let i = 0, 1, ..., 43. Specifically, this embodiment uses the mathematical software system SageMath, where the default primitive irreducible polynomial f(x) is generated as: f(x) = x 8 +x 4 +x 3 +x 2 +1.

[0082] The finite field GF(256) generated by F2 / (f(x)) can be expressed as:

[0083] GF(256)=F2 / (f(x))={a7x 7 +a6x 6 +...+a1x+a0|a i ∈{0,1}}.

[0084] Choose any primitive element x over the finite field GF(256) and generate several distinct elements, where each element λ i For polynomials of at most degree 7, the final generated λ0,λ1,...,λ 43 The polynomial form, binary, and hexadecimal representations are as follows: Figure 2 As shown.

[0085] Secondly, based on the generated elements, combined with the mapping defined above... Mapping is performed to obtain the kernel matrix for each code block. Specifically, let... Mapping according to the above definition Obtain the kernel matrix a∈[2], b∈[5], t=r, that is:

[0086]

[0087]

[0088] Finally, the corresponding parity check matrix is ​​determined based on the kernel matrix, thus completing the generation of the system MSR code. Specifically, the parity check matrix is ​​defined based on the kernel matrix as follows:

[0089]

[0090] Taking H4 as an example, according to the above definition, H4 can be:

[0091]

[0092] Based on the definition of the parity check matrix above, the specific value of each matrix is ​​calculated. Taking the H4 matrix as an example, the value of the H4 matrix is ​​calculated using the mathematical software system SageMath, as follows: Figure 3 As shown.

[0093] Example 2

[0094] Based on the system MSR code generation method proposed above, this embodiment also provides a system MSR code repair method with the minimum number of packets, including:

[0095] The information nodes of the (n=14, k=10, d=13) system MSR code generated in the above embodiment are repaired. The faulty nodes are repaired according to the different cases of the faulty nodes.

[0096] First, let's define node repair. Assume one node, C, has encountered an error. i , i∈

[10] , auxiliary node is C j , j∈

[14] \{i}, since the node repair of the MSR code needs to be downloaded from each auxiliary node A symbol, therefore D j→i Designed as a 4×16 matrix, its meaning is that when the i-th node encounters an error, the j-th node chooses to download which data portion to transmit to the erroneous node C. i That is, from auxiliary node C j The transmitted data can be represented as D j→i C j Therefore, the (n-1)β = 52 symbols downloaded from all auxiliary nodes can be represented as:

[0097]

[0098] Furthermore, the faulty node C i After obtaining the transmitted data, the repair matrix R is used based on this data. i Repairing erroneous blocks to restore the original data can be represented as:

[0099]

[0100] The above method can be used to repair the erroneous block.

[0101] Secondly, depending on the specific node where the error occurred, the faulty node should be repaired accordingly.

[0102] (1) First case

[0103] When any of the first b-1 nodes in group a fails, i.e., the node that fails is C. 5a+b When a∈[2], b∈[4], the set of helper nodes is If |H| = 13, then the faulty node can be {R} a,b C j :j∈H} Repair.

[0104] Furthermore, according to the formula We can obtain:

[0105]

[0106] Among them, e i This represents a row vector of length 4, where the i-th bit is 1 and the rest are 0, i = 0, 1, 2, 3; R a,b This indicates that the b-th node in group a selects the data to be transmitted to the erroneous node.

[0107] When an error occurs at C0, according to the above R... 0,0 It can be seen that each help node C j Pass the 1st, 5th, 9th, and 13th symbols, i.e., C, to C0. j (0),C j (4),C j (8),C j (12). The rest are similar, as detailed below. Figure 4 As shown, when any of the first four nodes C0 to C3 in the first group encounters an error, each helper node C... j Four discontinuous symbols are passed to the faulty node for repair, involving a total of four I / O operations. Therefore, C0, C1, C2, and C3 store relatively important data.

[0108] like Figure 5As shown, when any of the first four nodes C5 to C8 in the second group encounters an error, each helper node C... j Four consecutive symbols are sent to the faulty node for repair, which involves only one I / O operation. Therefore, C5, C6, C7, and C8 can store the most important data.

[0109] (2) Second case

[0110] When any of the last nodes in group a (i.e., C4, C9) encounters an error, the set of helper nodes is: |H|=13, let:

[0111] H1=H∩(5a+[5]),a∈[2];

[0112] H2=H\(5a+[5],a∈[2]);

[0113] Then the last node C of group a 5a+4 Can be {R a,b C 5a+z :5a+z∈H1,b∈[4]} and {R a,b C i :i∈H2,b∈[4]} Repair, that is, the helper node in the same group passes data {R} to the erroneous node. a,b C i :i∈H2},b∈[4], different groups of nodes pass data {R to the faulty node a,b C 5a+Z :5a+z∈H1,b∈[4]}.

[0114] Specifically, when an error occurs in C4, the data transmitted by each node to C4 is as follows:

[0115]

[0116]

[0117] When an error occurs in C9, the data transmitted by each node to C9 is as follows:

[0118]

[0119] Through the above method, this embodiment constructs and generates a Systematic MSR Code with the minimum number of packets. Compared to the existing General MSR Code, the Systematic MSR Code proposed in this embodiment further optimizes storage and data repair. Specifically, the Systematic MSR Code proposed in this embodiment can store not only redundant encoded data but also directly stored raw data. The Systematic MSR Code can achieve optimal repair with any k nodes, while other verification nodes are not considered for optimal repair. Since all information is stored in the information nodes, users only need to obtain data from the information nodes when downloading data, regardless of the verification nodes. Therefore, in a practical distributed storage system, only the rapid repair of the information nodes needs to be considered. Unlike the existing General MSR Code, which considers optimal repair of all nodes, the Systematic MSR Code proposed in this embodiment only focuses on the optimal repair of 10 information nodes, ignoring the repair issues of the 4 verification nodes, thereby optimizing storage and data repair.

[0120] More importantly, compared to existing (n=14, k=10) MSR code constructions, its packet count is significantly reduced. The MSR code proposed in this embodiment can achieve a smaller number of packets with the same parameters: since this embodiment only considers the optimal repair of 10 information nodes, the number of packets is... This embodiment reduces the number of packets, thereby lowering coding complexity and effectively reducing I / O overhead, further optimizing overall performance. Furthermore, the system MSR code proposed in this embodiment only requires downloading a small amount of data from other nodes during repair, recovering the data of failed nodes through linear combination. It has minimal repair bandwidth and high efficiency in repairing a single failed node, optimizing storage redundancy and multi-node repair.

[0121] Furthermore, utilizing the proposed system MSR code can improve system scalability: In large-scale storage systems, the MSR code can handle scenarios with more node failures while ensuring data reliability, which is crucial for systems with high scalability requirements; energy efficiency optimization: Since the MSR code reduces the bandwidth required for data transmission and repair, it can reduce the energy consumption of the system during data recovery, especially in large-scale data centers; more flexible design and implementation: The MSR code can be designed with different parameters to achieve a flexible trade-off between storage overhead, repair bandwidth, and repair complexity, adapting to the needs of different application scenarios.

[0122] Example 3

[0123] This embodiment provides an electronic device, including: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the method provided in this embodiment.

[0124] Example 4

[0125] This embodiment also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, will cause the processor to execute the method described above in this embodiment.

[0126] Example 5

[0127] This embodiment provides a computer program product including executable instructions, which are computer instructions; the executable instructions are stored in a computer-readable storage medium. When the processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the electronic device performs the method described in this embodiment.

[0128] The steps and methods involved in Embodiments 2 to 5 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0129] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0130] The above description is only a preferred embodiment of the present invention. Although the specific implementation of the present invention has been described in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the present invention.

Claims

1. A method for generating MSR codes in a system with a minimum number of packets, characterized in that, include: For by n A distributed storage system consisting of nodes is defined. The structure of the system MSR code; wherein, the system MSR code has a total of n individual code blocks ,before k Each code block serves as an information node, with the remaining code blocks as the next... nk Each code block serves as a check node, storing all information in the information node and performing checks. k Each information node is grouped, at which point the system's MSR code is divided into packets. ; Based on the defined structure of the systematic MSR code, it is calculated and generated in a finite field. The system MSR code is: using the default generated primitive irreducible polynomial. In a finite field Choose any primitive element, use it to generate several different elements, and perform a mapping based on the generated elements to obtain each code block. kernel Matrix, and according to kernel The matrix determines the corresponding parity check matrix, thereby completing the generation of the system's MSR code; Among them, a vector of elements Mapping to a finite field The size of the top is matrix The mapping is represented as: ； in, , It is a positive integer. for A block matrix, where each block is of length . Column vectors; for , The Line number The column is represented as: ； In the above formula, ,element ; Using the default generated primitive irreducible polynomial In a finite field Choose any one primitive element and use it to generate several different elements, including: Using primitive irreducible polynomials ,pass Generating finite fields , represented as: ； In a finite field Choose one of the fundamental elements Using this primitive element, several different elements are generated, and the elements are set. Each element It is a polynomial with a maximum degree of 7; according to kernel The matrix determines the corresponding parity check matrix, which is defined as follows: ； In the above formula, It is the identity matrix. Indicates the Kronecker product. This represents the Kronecker product of the elements in matrices A and B. a Indicates the number of groups. b Indicates the first b Information nodes.

2. The system MSR code generation method with minimum packet number as described in claim 1, characterized in that, generate The system MSR code, the generated system MSR code has a minimum number of packets. , , Indicates the number of nodes that helped with the repair process; before k=10 One code block is used as an information node, and the remaining four code blocks are used as check nodes. All information is stored in the information nodes, and the ten information nodes are grouped together.

3. A system MSR code repair method with minimum number of packets, characterized in that, include: For those generated using the method described in any one of claims 1-2 The system uses MSR codes to repair information nodes. Depending on the specific node that erroneous, the repair process is tailored to different scenarios, including: when a The first in the group b When any of the -1 nodes encounters an error, the set of helper nodes is: Then the faulty node is repair; when a When any node in the last node of the group encounters an error, the set of helper nodes is: ,set up: ； ； Then the first The last node of the group quilt and repair.

4. The system MSR code repair method with minimum packet count as described in claim 3, characterized in that, include: When the first 4 nodes of the first group When any node in the process encounters an error, each helper node... To repair the erroneous node, pass four discontinuous symbols. When the first 4 nodes of the second group When any node in the process encounters an error, each helper node... Send four consecutive symbols to the faulty node to repair it.

5. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the system MSR code generation method with minimum number of packets as described in any one of claims 1-2.

6. A computer-readable storage medium, characterized in that, The system stores executable instructions that, when executed by a processor, implement the system MSR code generation method with minimum number of packets as described in any one of claims 1-2.

7. A computer program product, characterized in that, The computer program product includes executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, it implements the system MSR code generation method with minimum number of packets as described in any one of claims 1-2.

Images