Data storage coding method, data restoration method, data storage coding device, data restoration device and storage medium

Abstract

The invention provides a data storage coding method and device, a data recovery method and device and a storage medium, and the method comprises the steps: obtaining a coding parameter of a maximum recoverable code; determining the coding code length, finite field parameters and storage parameters of the maximum repairable code according to the coding code length range; determining a check matrix of the maximum repairable code according to the finite field parameters and the finite projective geometry; according to the check matrix and the to-be-stored data, generating check data of the to-be-stored data; and storing the to-be-stored data and the verification data according to the coding length and the storage parameters. According to the invention, the finite field parameter of the maximum repairable code is relatively small and the code length is relatively long, so that the coding and decoding operation speed can be improved; in addition, the used finite field feature is 2, and only the exclusive-or operation is used when the failure node is repaired, so that the calculation complexity of node repair is reduced.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a data storage encoding method, a data repair method, and a device and storage medium thereof. Background Art

[0002] With the digital development of society, distributed storage systems have become the digital infrastructure of today's society. Generally speaking, redundancy strategies are needed to deal with software and hardware failures of storage systems. That is, when storing data, the original k data blocks are encoded to obtain n (n>k) data blocks, and the encoded n data blocks are stored in n storage nodes respectively to ensure that even if there are a small number of node failures, data will not be lost. In this case, the encoding method needs to meet the requirements of both ensuring the redundancy rate of the encoding and having the maximum fault tolerance as much as possible.

[0003] The prior art provides an ideal fault-tolerant coding scheme for distributed storage systems, namely a special maximally recoverable code (MRC). The MRC has the strongest fault tolerance capability under a given local repair group configuration, and has the advantages of low repair bandwidth overhead and high fault tolerance capability.

[0004] However, the existing extremely large repairable codes have the disadvantages of large finite fields and relatively short code lengths, resulting in slow computing speeds. Summary of the invention

[0005] The purpose of the present application is to provide a data storage encoding method, a data repair method, and a device and storage medium thereof to address the problem of slow operation speed of extremely large repairable codes in the prior art.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides a data storage encoding method, the method comprising:

[0008] Obtaining encoding parameters of the extremely large repairable code, the encoding parameters including: an encoding code length range;

[0009] Determining the encoding code length, finite field parameters, and storage parameters of the extremely large repairable code according to the encoding code length range;

[0010] Determining a check matrix of the maximal repairable code according to the finite field parameters and the finite projective geometry;

[0011] Generating verification data of the data to be stored according to the verification matrix and the data to be stored;

[0012] According to the encoding code length and the storage parameter, the data to be stored and the verification data are stored.

[0013] In a possible implementation, determining the encoding code length, finite field parameters, and storage parameters of the extremely large repairable code according to the encoding code length range includes:

[0014] Determine the finite field parameter and the encoding code length of the extremely large repairable code according to the encoding code length range and the preset linear relationship between the encoding code length and the finite field parameter, wherein the finite field parameter is a power of 2, and the linear relationship between the encoding code length and the finite field parameter is: q≤n≤4(q+1) / 3, and n is a multiple of 4, wherein n is the encoding code length, and q is the finite field parameter;

[0015] According to the finite field parameter of the extremely large repairable code and the encoding code length, the storage parameter of the extremely large repairable code is calculated.

[0016] In a possible implementation, determining the check matrix of the maximally repairable code according to the finite field parameters and the finite projective geometry includes:

[0017] According to the finite field parameters and the finite projective geometry PG(1,q), determine the set of projection points;

[0018] A check matrix of the maximal repairable code is determined according to the points in the projection point set.

[0019] In a possible implementation, determining the check matrix of the maximal repairable code according to the points in the projection point set includes:

[0020] Corresponding the points in the projection point set to two-dimensional vectors in a two-dimensional linear space to obtain a linear equation system;

[0021] The linear equations are solved, and the solutions of the linear equations are substituted into an initial check matrix to obtain the check matrix of the maximally repairable code.

[0022] In a possible implementation, the linear equation system is a i =λ i，1 b i -λ i，2 c i , where a i , b i , c i is a projection point in the projection point set, λ i，1 ,λ i，2 is the unknown number that needs to be solved;

[0023] Substituting the solution of the linear equations into an initial check matrix to obtain the check matrix of the maximally repairable code includes:

[0024] If the solution of the linear equations is (c i，1 , c i，2 ),make and will Substitute into the initial check matrix to obtain the check matrix.

[0025] In a possible implementation, the first set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 8;

[0026] The second set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 16;

[0027] The third set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 32;

[0028] The fourth group of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 64;

[0029] The fifth set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 128;

[0030] The sixth set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 256;

[0031] The seventh group of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 512;

[0032] The second set of values ​​of the extremely large repairable code includes: code length Dimensionality The locality parameter r=3, the finite field parameter q=1024, where Is a positive integer.

[0033] In a second aspect, another embodiment of the present application provides a data repair method, the method is used to repair the data to be stored and the verification data stored in the first aspect, wherein the data to be stored and the verification data are divided into multiple data groups, each of the data groups includes at least one data to be stored and at least one verification data, each of the data groups is stored in multiple storage nodes, the method includes:

[0034] Determine a node to be repaired among the multiple storage nodes and a target data group where the node to be repaired is located;

[0035] Determine a plurality of target storage nodes corresponding to the target data group;

[0036] R target storage nodes are taken out from the multiple target storage nodes, and data of the r target storage nodes are XOR-processed to repair the node to be repaired.

[0037] In a possible implementation, the method further includes:

[0038] Determine a plurality of nodes to be repaired among a plurality of storage nodes and a target data group where each of the nodes to be repaired is located;

[0039] If each node to be repaired and the target data group where each node to be repaired is located meet the preset conditions, k storage nodes are taken out from the multiple storage nodes, and the data in the k storage nodes are decoded by using a check matrix to obtain a data decoding result.

[0040] In a third aspect, another embodiment of the present application provides a data storage encoding device, wherein the data storage encoding device is used to:

[0041] Obtaining encoding parameters of the extremely large repairable code, the encoding parameters including: an encoding code length range;

[0042] Determining the encoding code length, finite field parameters, and storage parameters of the extremely large repairable code according to the encoding code length range;

[0043] Determining a check matrix of the maximal repairable code according to the finite field parameters and the finite projective geometry;

[0044] Generating verification data of the data to be stored according to the verification matrix and the data to be stored;

[0045] According to the encoding code length and the storage parameter, the data to be stored and the verification data are stored.

[0046] In a fourth aspect, another embodiment of the present application provides a data repair device, the data repair device is used to repair the data to be stored and the verification data stored in the first aspect, wherein the data to be stored and the verification data are divided into multiple data groups, each of the data groups includes at least one data to be stored and at least one verification data, each of the data groups is stored in multiple storage nodes, and the data repair device is used to:

[0047] Determine a node to be repaired among the multiple storage nodes and a target data group where the node to be repaired is located;

[0048] Determine a plurality of target storage nodes corresponding to the target data group;

[0049] R target storage nodes are taken out from the multiple target storage nodes, and data of the r target storage nodes are XOR-processed to repair the node to be repaired.

[0050] In a fifth aspect, another embodiment of the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspect or the second aspect are executed.

[0051] In the sixth aspect, another embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program code. When the computer program code is executed by a computing device, the computing device executes the steps of the method described in any one of the first aspect or the second aspect.

[0052] The beneficial effects of the present application are: by obtaining the coding parameters of the extremely large repairable code, the coding code length, finite field parameters and storage parameters of the extremely large repairable code are determined according to the coding code length range, and the check matrix of the extremely large repairable code is determined according to the coding code length, finite field parameters and finite projective geometry, on this basis, the check data of the data to be stored is generated according to the check matrix, and the data to be stored and the check data are stored according to the coding code length and storage parameters. The finite field parameters of the extremely large repairable code can be made smaller and the code length longer, so that the operation speed can be improved, the fast storage of the data to be stored and the check data can be realized, and the storage performance can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 A schematic diagram of a check matrix H of a very large repairable code provided in an embodiment of the present application;

[0055] Figure 2 A schematic diagram of a flow chart of a data storage encoding method provided in an embodiment of the present application;

[0056] Figure 3 A storage schematic diagram of a distributed storage system obtained by executing the data storage encoding method provided in an embodiment of the present application;

[0057] Figure 4 A schematic diagram for executing the data storage encoding method provided in an embodiment of the present application;

[0058] Figure 5 A schematic diagram of a flow chart for determining the encoding code length, finite field parameters and storage parameters of a very large repairable code in the data storage encoding method provided in an embodiment of the present application;

[0059] Figure 6 A schematic diagram of a flow chart for determining a check matrix of a very large repairable code in a data storage coding method provided in an embodiment of the present application;

[0060] Figure 7 Another schematic diagram of a flow chart for determining a check matrix of a very large repairable code in the data storage coding method provided in an embodiment of the present application;

[0061] Figure 8 A schematic diagram of a check matrix H of a very large repairable code obtained by executing the data storage encoding method provided in an embodiment of the present application;

[0062] Fig. 9 A schematic diagram of a process for generating verification data for data to be stored in the data storage encoding method provided in an embodiment of the present application;

[0063] Fig.10 A schematic diagram of a process for storing data to be stored and verification data in the data storage encoding method provided in an embodiment of the present application;

[0064] Fig.11 A schematic diagram of a process flow of a data repair method provided in an embodiment of the present application;

[0065] Fig.12 A flowchart of a data decoding method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0067] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0068] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0069] By adopting a distributed storage solution, massive amounts of data can be stored in a large number of distributed data storage nodes, so that efficient data reading and data calculation can be performed from these distributed storage nodes. In order to prevent data loss in a distributed storage system, it is usually necessary to perform redundant processing on massive amounts of data so as to encode the massive amounts of data, so that the massive amounts of data can be recovered by decoding after partial data loss. The prior art provides an ideal fault-tolerant coding scheme for distributed storage systems, namely a special maximally recoverable code (MRC), which has the strongest fault tolerance capability under a given local repair group configuration, and has the advantages of low repair bandwidth overhead and high fault tolerance capability. The maximally recoverable code is an ideal fault-tolerant coding scheme for distributed storage systems.

[0070] However, since the size of the finite field is related to the speed of finite field operations, generally speaking, the larger the finite field, the slower the operation speed. Therefore, the extremely large repairable code in the prior art is limited by the code length, and there is a problem of slow operation speed due to the large finite field and the short code length.

[0071] Based on the above problems, the embodiment of the present application proposes a data storage coding method, which obtains the coding parameters of the extremely large repairable code, thereby determining the coding code length, finite field parameters and storage parameters of the extremely large repairable code according to the coding code length range, and determines the check matrix of the extremely large repairable code according to the coding code length, finite field parameters and finite projective geometry. On this basis, according to the check matrix, the check data of the data to be stored is generated, and according to the coding code length and storage parameters, the data to be stored and the check data are stored. The finite field parameters of the extremely large repairable code can be made smaller, thereby improving the operation speed, realizing the rapid storage of the data to be stored and the check data, and improving the storage efficiency.

[0072] First, the relevant concept definitions involved in the data storage encoding method provided in the embodiment of the present application are explained.

[0073] Definition 1: Let F q is a finite field with q elements, a linear code of length n Yes F q Upper linear space A non-empty subspace of . The dimension k of the code is called The dimension of , n is called the code length. The vector c in (c0, c1, ..., c n-1 ) is called code Codewords, where c i ∈F q (i=0,1,…,n-1).

[0074] Definition 2: Coding is a finite field F q A linear code on the linear space Linear code The dual code of Right now

[0075]

[0076] Definition 3: Let x and y be fields F q The relative distance between the two vectors of length n is d(x, y), which is defined as the number of all different components in x and y. That is, if x = (x0, x1, ..., x n-1 ), y=(y0,y1,…,yn-1 ),but

[0077] d(x,y)=d(x0,y0)+…+d(x n-1 ,y n-1 )

[0078] in

[0079]

[0080] Definition 4: Assume is a finite field F q A linear code on which at least two code words are contained. The minimum distance Defined as:

[0081]

[0082] Definition 5: For a [n, k, d] linear code If a codeword can be restored after being deleted by accessing at most r other codewords, then the codeword is said to have locality r.

[0083] Definition 6: For a [n, k, d] linear code If every codeword has locality r, then the codeword is called is a Locally Repairable Code (LRC) with a locality parameter of r. For any symbol position i∈[n], the set consisting of the position of the symbol and the positions of other symbols to be accessed to repair the symbol is called the local repair group of position i, denoted as And |L i |≤r+1.

[0084] Definition 7: Let F q is a finite field with q elements. Assume that m ≥ 2 and the point set Ω contains The non-zero point

[0085] (a0, a1, ..., a m ), a i ∈F q

[0086] And it is stipulated that (a0, a1, ..., a m ) and (λa0, λa1, ..., λa m ) is a point in the same Ω, where λ is F q The non-zero elements in Ω are called the secondary coordinates of the point. The Chinese Communist Party m+1-1 non-zero points, each of which is the same point as q-1 points in Ω, so there are points. Passing through two points (a0, a1, ..., a m ) and (b0, b1, ..., b m ) contains the following points:

[0087] (λa0+μb0, λa1+μb1,…, λa m +μb m )

[0088] where λ,μ∈F q , and they are not zero at the same time. A line contains q+1 points, because λ and μ have q 2 -1 choices, each point is the same point as q-1 points in Ω.

[0089] The point set defined in the above manner is a finite projective geometry, and this finite projective geometry is denoted as PG(m,q).

[0090] On this basis, the relevant concepts and conclusions of extremely large repairable codes are explained.

[0091] Definition 8: A maximally repairable code is a local repair code whose local repair groups are mutually disjoint and whose sizes are equal. Its check matrix can be transformed into the form of matrix H through equivalent transformation, where Figure 1 A schematic diagram of the check matrix H of the extremely large repairable code provided in the embodiment of the present application, referring to Figure 1 As shown, the local repair code determined by the check matrix H is There are local repair groups, global checksum rows, for any i∈[l], j∈[r], 0 means The first l rows of the check matrix H are local check rows, each with a weight of r+1, and the supports of each row do not intersect each other, which ensures the local repair property of the maximally repairable code. The last h rows of the check matrix H are called global check rows, which together with the local check rows ensure the maximally repairable property of the maximally repairable code.

[0092] The above explains the relevant conceptual definitions involved in the data storage encoding method provided in the embodiment of the present application. It can be understood that the data storage encoding method provided in the embodiment of the present application can be applied to the scenario of storing the data to be stored in a distributed storage system. By executing the data storage encoding method provided in the embodiment of the present application, the data to be stored can be redundantly encoded through the extremely large repairable code constructed in the embodiment of the present application, and stored in the distributed storage system.

[0093] At the same time, after the data to be stored is stored using the data storage encoding method provided in the embodiment of the present application, since it is encoded with a highly repairable code, when data is missing in the distributed system, the missing data can also be restored, thereby ensuring the integrity of the stored data.

[0094] In addition, after the data to be stored is stored using the data storage encoding method provided in the embodiment of the present application, since a large repairable code is used for encoding, the data stored in the distributed system can also be verified, thereby ensuring the accuracy of the stored data.

[0095] It is worth noting that there may be several nodes capable of storage in the distributed storage system described in the embodiment of the present application, and each node may be a server or a process running on a server, and the nodes are interconnected through a network to form a distributed storage system. When storing the data to be stored in the distributed storage system, it is first possible to determine which nodes or nodes in the distributed storage system the data to be stored is to be stored, that is, the storage nodes are first determined, and on this basis, by executing the data storage encoding method provided in the embodiment of the present application, the storage nodes are divided into data nodes, local check nodes, and global check nodes, and a local repair group is obtained, and the data to be stored is processed by the extremely large repairable code provided in the embodiment of the present application, and the obtained data is stored in the storage node, specifically, the processed data for verification is stored in the corresponding local check node or global check node, and the data to be stored is stored in the data node, so that redundant storage of the data to be stored can be realized, so that the distributed storage system has a certain fault tolerance when storing data, and at the same time, the data stored in the distributed storage system can be repaired and verified.

[0096] It can be understood that in the process of storing the data to be stored in the storage node, the data to be stored can be divided into multiple strips, and the data of each stripe can be stored in the storage node by executing the data storage encoding method provided in the embodiment of the present application.

[0097] The data storage encoding method provided in the embodiments of the present application is described in detail below in combination with multiple embodiments.

[0098] Figure 2 A schematic diagram of a data storage encoding method provided in an embodiment of the present application, referring to Figure 2 As shown, the execution subject of the data storage encoding method can be any electronic device with processing capabilities, and the method includes:

[0099] S201. Obtain coding parameters of a very large repairable code.

[0100] The encoding parameters include: encoding code length range.

[0101] It can be understood that in the data storage encoding method provided in the embodiment of the present application, before redundant encoding is performed on the data to be stored, it is necessary to determine the check matrix of the extremely large repairable code provided in the present application to describe the extremely large repairable code provided in the present application, so that the data to be stored can be encoded and stored. When determining the check matrix of the extremely large repairable code provided in the embodiment of the present application, the encoding parameters of the extremely large repairable code can be obtained first to constrain the extremely large repairable code, so as to ensure the extremely large repairable property of the extremely large repairable code.

[0102] Optionally, the coding parameters of the extremely large repairable code are obtained to constrain the extremely large repairable code, wherein the coding parameters include a coding code length range. The coding parameters are used to indicate the constraints that each parameter in the extremely large repairable code provided in the present application should satisfy. Specifically, the coding code length range is used to indicate the constraint range that the coding code length n should satisfy in the extremely large repairable code provided in the present application, that is, when storing the data to be stored in multiple storage nodes, the number of storage nodes should satisfy the constraint range.

[0103] Optionally, the coding parameters may also include a locality parameter r, a global check bit h, and a minimum distance d. The locality parameter r is used to indicate the constraint range that the locality parameter needs to satisfy in the extremely large repairable code provided in the present application, that is, the minimum number of other nodes or symbols that need to be visited in order to repair a failed node or symbol. The global check bit h is used to indicate the constraint conditions that the global check bit needs to satisfy in the extremely large repairable code provided in the present application. The minimum distance d is used to indicate the minimum value of the number of different bits between two codewords.

[0104] S202: Determine the encoding code length, finite field parameters, and storage parameters of the extremely large repairable code according to the encoding code length range.

[0105] It can be understood that after obtaining the encoding code length range of the extremely large repairable code, before determining the check matrix of the extremely large repairable code, it is also necessary to determine the various parameters of the extremely large repairable code, including the encoding code length, finite field parameters and storage parameters of the extremely large repairable code.

[0106] The encoding code length n is the encoding length of the extremely large repairable code, that is, the number of storage nodes required to store the data to be stored. The finite field parameter is q in the above definition, and the storage parameters include the dimension k and the local repair group parameter l. The dimension k is the number of bits of data in a stripe, that is, there are k data nodes among n storage nodes, which are used to store the original data. The local repair group parameter l is the number of local repair groups of the extremely large repairable code.

[0107] Optionally, after obtaining the encoding code length range, the encoding code length, finite field parameters and storage parameters of the extremely large repairable code can be determined by calculation in combination with the relationship between the preset encoding code length and the finite field parameters and storage parameters.

[0108] S203: Determine a check matrix of the maximal repairable code according to finite field parameters and finite projective geometry.

[0109] It can be understood that the process of determining the check matrix of the extremely large repairable code is the process of determining the check matrix H provided in the above definition 8. After obtaining the finite field parameter q, the finite projective geometry in the above definition 7 can be combined to perform calculations to obtain the check matrix of the extremely large repairable code. The check matrix is ​​used to reflect the local repair characteristics of the extremely large repairable code. Specifically, the structure of the rows and columns of the check matrix defines the local repair group, and the rows and columns in the check matrix can be used to generate check data.

[0110] Optionally, a finite projective geometry PG(1,q) can be pre-constructed, and the projection points in the finite projective geometry can be determined based on the finite field parameter q, so that a system of linear equations can be obtained by treating the projection points as vectors on the finite field, and the system of linear equations can be solved to determine the check matrix of the maximally repairable code.

[0111] S204: Generate verification data of the data to be stored according to the verification matrix and the data to be stored.

[0112] It can be understood that after obtaining the check matrix H, the extremely large repairable code provided by the embodiment of the present application is determined, that is, after obtaining the check matrix H, the values ​​of the rows and columns in the check matrix H used to check the data to be stored can be obtained, then the data to be stored can be processed according to the check matrix H to generate the check data of the data to be stored, so that the check data of the data to be stored can be stored in the storage node. Among them, the check data of the data to be stored includes the local check data of the data to be stored and the global check data of the data to be stored.

[0113] Optionally, the check matrix H may be calculated to obtain a generator matrix M of the maximally repairable code, and the data to be stored may be calculated based on the generator matrix M to obtain check data of the data to be stored.

[0114] Optionally, the data to be stored may be calculated according to the check matrix H to obtain check data of the data to be stored.

[0115] S205. Store the data to be stored and the verification data according to the encoding code length and the storage parameters.

[0116] It can be understood that after obtaining the verification data of the data to be stored, it is necessary to store the data to be stored and the verification data of the data to be stored in the storage node. Specifically, the data to be stored is stored in the data node, and the verification data of the data to be stored is stored in the local verification node and the global verification node.

[0117] Optionally, after obtaining the verification data of the data to be stored, the data to be stored and the verification data may be stored in corresponding nodes in the storage nodes according to the encoding code length and the storage parameters.

[0118] For example, Figure 3 A storage schematic diagram of a distributed storage system obtained by executing the data storage encoding method provided in an embodiment of the present application, referring to Figure 3 As shown, the distributed storage system includes a network core and 16 data nodes, wherein D1-D10 are data nodes, D1-D10 are used to store data to be stored, L1-L4 are local check nodes, L1-L4 are used to store local check data generated by extremely large repairable codes, P1-P2 are global check nodes, P1-P2 are used to store global check data generated by extremely large repairable codes. By executing the data storage encoding method provided in the embodiment of the present application, it is possible to determine that the encoding code length n=16, and 301-304 are the four local repair groups determined at this time.

[0119] For example, Figure 4 A schematic diagram of the method for encoding data storage provided in the embodiment of the present application is shown in FIG. Figure 4 As shown, assuming that the data of the current stripe is d1-d10, M is the generating matrix obtained by the check matrix H, by executing the data storage encoding method provided in the embodiment of the present application, local check data l1-l4 can be obtained, and global check data p1-p2 can be obtained, and d1-d10 is stored in the data node D1-D10, the local check data l1-l4 is stored in the local check node L1-L4, and the global check data p1-p2 is stored in the global check node P1-P2.

[0120] In this embodiment, by obtaining the encoding parameters of the extremely large repairable code, the encoding code length, finite field parameters and storage parameters of the extremely large repairable code are determined according to the encoding code length range, and the check matrix of the extremely large repairable code is determined according to the encoding code length, finite field parameters and finite projective geometry. On this basis, the check data of the data to be stored is generated according to the check matrix, and the data to be stored and the check data are stored according to the encoding code length and storage parameters. The finite field parameters of the extremely large repairable code can be made smaller and the code length longer, so that the operation speed can be improved, the fast storage of the data to be stored and the check data can be realized, and the storage performance is improved.

[0121] Figure 5 A schematic diagram of a flow chart for determining the encoding code length, finite field parameters and storage parameters of a very large repairable code in the data storage encoding method provided in an embodiment of the present application.

[0122] In one possible implementation, referring to Figure 5 As shown, in the above step S202, when determining the encoding code length, finite field parameters and storage parameters of the extremely large repairable code according to the encoding code length range, the following steps can be performed, specifically including:

[0123] S501. Determine the finite field parameters and the encoding code length of the extremely large repairable code according to the encoding code length range and the preset linear relationship between the encoding code length and the finite field parameter.

[0124] Among them, the finite field parameter is a power of 2, and the linear relationship between the encoding code length n and the finite field parameter q is: q≤n≤4(q+1) / 3 and n is a multiple of 4.

[0125] Optionally, after obtaining the encoding code length range, the encoding code length range and the linear relationship between the preset encoding code length and the finite field parameter can be calculated to determine the optimal finite field parameter, and after obtaining the finite field parameter, the encoding code length can be calculated based on the linear relationship between the finite field parameter q and the encoding code length n.

[0126] Exemplarily, after obtaining the range of the encoding code length n, let the finite field parameter q be a power of 2, and 3|(q+1), and at the same time, let n=4(q+1) / 3, determine the finite field parameter q, and further determine the encoding code length n.

[0127] It can be understood that since the electronic device uses binary to store and calculate the data to be stored, the finite field parameter q is a power of 2, which can improve the calculation speed and ensure the performance of data storage.

[0128] S502: Calculate storage parameters of the extremely large repairable code according to the finite field parameters of the extremely large repairable code and the encoding code length.

[0129] Optionally, after obtaining the finite field parameters and the encoding code length of the extremely large repairable code, the storage parameters of the extremely large repairable code may be calculated, wherein the storage parameters include the dimension k and the local repair group parameter l.

[0130] Exemplarily, the dimension k=nq-3, where n is the encoding code length and q is the finite field parameter. The local repair group parameter l=(q+1) / 3, where q is the finite field parameter.

[0131] Through the range of encoding code length and the linear relationship between the preset encoding code length and the finite field parameter, the finite field parameter and the encoding code length of the maximally repairable code are determined, and based on the finite field parameter and the encoding code length of the maximally repairable code, the storage parameter of the maximally repairable code is calculated, which can make the obtained finite field parameter as small as possible, thereby improving the calculation speed, realizing the fast storage of the data to be stored and the verification data, and improving the storage performance.

[0132] Figure 6 A schematic diagram of a flow chart for determining a check matrix of a very large repairable code in a data storage coding method provided in an embodiment of the present application.

[0133] In one possible implementation, referring to Figure 6 As shown, in the above step S203, when determining the check matrix of the maximal repairable code according to the finite field parameters and the finite projective geometry, the following steps may be performed, specifically including:

[0134] S601. Determine a set of projection points according to finite field parameters and finite projective geometry.

[0135] Optionally, according to the finite field parameter q, a partition of a finite projective geometry PG(1,q) is randomly selected, satisfying the following condition ∪S i =PG(1,q)(i=1,...,l),|S i |=3, and we get the projection point set ∪S i .

[0136] S602: Determine a check matrix of the maximal repairable code according to the points in the projection point set.

[0137] Optionally, after obtaining the projection point set, the points in the projection point set may be matched with the two-dimensional linear space and solved, thereby obtaining a check matrix of the extremely large repairable code.

[0138] Figure 7 Another schematic diagram of a flow chart for determining a check matrix of a very large repairable code in the data storage coding method provided in an embodiment of the present application.

[0139] In one possible implementation, referring to Figure 7 As shown, in the above step S602, when determining the check matrix of the maximal repairable code according to the points in the projection point set, the following steps may be performed, including:

[0140] S701. Correspond the points in the projection point set to two-dimensional vectors in a two-dimensional linear space to obtain a system of linear equations.

[0141] Optionally, the set of projection points ∪S i =PG(1,q)(i=1,...,l),|Si |=3, assuming S i ={a i , b i , c i}, according to the projection point a i , b i , c i The one-to-one correspondence with the two-dimensional linear space can be a i , b i , c i Considered as F q The 2D vector on the graph gives the following linear equations:

[0142] a i =λ i，1 b i -λ i，2 c i

[0143] Among them, λ i，1 ,λ i，2 is the unknown number that needs to be solved.

[0144] S702, solving the linear equations, substituting the solutions of the linear equations into the initial check matrix, and obtaining the check matrix of the maximally repairable code.

[0145] Optionally, the above linear equations are solved. Exemplarily, a solution of the linear equations can be (c i，1 , c i，2 ),make and will Substitute into the check matrix H in the above definition 8. Specifically, the last two rows of the check matrix H can be substituted to obtain the check matrix of the maximally repairable code.

[0146] For example, Figure 8 A schematic diagram of a check matrix H of a very large repairable code obtained by executing the data storage encoding method provided in an embodiment of the present application, referring to Figure 8 As shown, it is the check matrix H of the maximal repairable code on F8 [coding code length n=12, dimension k=6, minimum distance d=4, locality parameter r=3]. Wherein, w is the primitive element of F8.

[0147] In one possible implementation, the linear system is a i =λ i，1 b i -λ i，2 c i , where a i , b i , c i is a projection point in the projection point set, λ i，1 ,λi，2 is the unknown number that needs to be solved.

[0148] In the above step S702, the solution of the linear equation system is substituted into the initial check matrix to obtain the check matrix of the maximally repairable code, including:

[0149] If the solution of the linear equation system is (c i，1 , c i，2 ),make and will Substitute into the initial check matrix to obtain the check matrix.

[0150] Exemplarily, the construction process of the check matrix H of the maximal local repair code with a code length of q≤n≤4(q+1) / 3 and n being a multiple of 4 in the data storage coding method provided in an embodiment of the present application is described.

[0151] Step 1: Get the input q, where q is a power of 2.

[0152] Step 2: Get input l, which is required to be an integer and q / 4≤l≤(q+1) / 3.

[0153] Step 3: Continue to refer to Figure 1 As shown, the initial check matrix H is configured. Specifically, there are l local repair groups, the locality parameter r = 3, and the global check rows h = 2. The local check row part of H is determined according to the number of local repair groups l and the locality parameter r.

[0154] Step 4. Randomly select a subset S of PG(1,q) containing 3l elements. Randomly select a partition of S ∪S i =PG(1,q)(i=1,...,l), and |S i |=3.

[0155] Step 5: Start looping, i=1, 2, ..., l. In each loop:

[0156] Let S i ={a i , b i , c i}, according to the one-to-one correspondence between the projection point and the two-dimensional linear space, a i , b i , c i Considered as F q There are three two-dimensional vectors on . There are the following linear equations:

[0157] a i =λ i，1 b i -λ i，2 c i

[0158] Among them, λ i，1 ,λ i，2 It is an unknown.

[0159] Suppose a solution of the linear equations is (c i，1 , c i，2 ), the global check part of the i-th local repair group of the check matrix H is:

[0160]

[0161] After the loop is executed, the check matrix H is obtained.

[0162] Optionally, by executing the data storage encoding method provided in the embodiment of the present application, the parameter values ​​of the maximum repairable code can be obtained as shown in Table 1:

[0163] Table 1 Parameter value table of extremely large repairable code

[0164]

[0165] Where l is the aforementioned local repair group parameter, that is, the number of local repair groups of the maximum repairable code, is a positive integer. [a,b] is the set of integers from a to b, that is, [a,b] = {a,a+1,…,b}.

[0166] Fig. 9 A schematic diagram of a process for generating verification data for data to be stored in the data storage encoding method provided in an embodiment of the present application.

[0167] In one possible implementation, referring to Fig. 9 As shown, in the above step S204, when generating the check data of the data to be stored according to the check matrix and the data to be stored, the following steps can be performed, specifically including:

[0168] S901. Determine a generator matrix of the extremely large repairable code according to the check matrix of the extremely large repairable code.

[0169] Optionally, after obtaining the check matrix H of the extremely large repairable code, a matrix operation may be performed on the check matrix H of the extremely large repairable code to obtain a generator matrix of the extremely large repairable code.

[0170] S902: Generate verification data according to the generation matrix and the data to be stored.

[0171] Optionally, after the extremely large repairable code generates the matrix M, the data to be stored may be operated according to the generating matrix M, so as to obtain local verification data of the data to be stored and global verification data of the data to be stored according to the generating matrix M.

[0172] In a possible implementation, determining the generator matrix of the extremely large repairable code according to the check matrix of the extremely large repairable code in step S901 may include:

[0173] According to the check matrix of the extremely large repairable code, a dual matrix of the check matrix is ​​determined.

[0174] Optionally, after obtaining the check matrix H of the maximally repairable code, calculate the dual matrix H of the check matrix H * .

[0175] According to the dual matrix, the generator matrix of the maximal repairable code is determined.

[0176] Optionally, get the dual matrix H * After that, the dual matrix H * Perform elementary row transformation and column permutation to transform the dual matrix H * The first generator matrix G is transformed, where the first generator matrix G is in the form of a unit matrix on the left side I, which can be described as G = [I|M]. The second generator matrix M is determined by the first generator matrix G, and the second generator matrix M is used as the generator matrix of the maximally repairable code.

[0177] The first generator matrix G can also be described as a generator matrix in a systematic form in coding theory. G has k rows and n columns. Through vector-matrix multiplication, M can generate the check bits of a very large repairable code, and each column of M can correspond to a check bit. The weight of the column is less than r, and there are 3 columns with weight greater than or equal to r. Take out the columns with weight less than r and any 1 column with weight greater than r in M, and put these The column is called the local check column, and the other two columns of M are called the global check columns.

[0178] In a possible implementation, generating verification data according to a generation matrix and data to be stored includes:

[0179] According to the encoding code length and storage parameters, the data to be stored is divided into multiple sub-data segments to be stored.

[0180] Exemplarily, for ease of processing, the data to be stored can be divided into a stripe of klog2(q) bits each in bit order as a sub-data segment to be processed, and each stripe, that is, each sub-data segment to be processed, can be divided into k groups, each group having log2(q) bits.

[0181] For each sub-data segment to be stored, the sub-data segment to be stored is encoded according to the generation matrix to obtain verification data of the sub-data segment to be stored.

[0182] Optionally, continue to refer to Figure 4As shown, after the generator matrix M is obtained, vector operations can be performed on the generator matrix and the data to be stored to obtain verification data of the data to be stored.

[0183] For example, each log2(q) bit in each stripe can be considered as a finite field F q An element on F q A row vector x on M. Calculate the check data y=xM, where the check data y is a row vector of length nk, and the check data y is used as the check bit after encoding. Specifically, when the check data y is calculated from the local check column of M, the bit is called the local check bit, and the other bits are called the global check bits.

[0184] Fig.10 A schematic diagram of a process for storing data to be stored and verification data in the data storage encoding method provided in an embodiment of the present application.

[0185] In one possible implementation, referring to Fig.10 As shown, in the above step S205, when storing the data to be stored and the verification data according to the encoding code length and the storage parameter, the following steps can be performed, specifically including:

[0186] S1001. Determine the length of each storage unit corresponding to the sub-data segment to be stored according to the encoding code length and storage parameters.

[0187] It can be understood that, for the convenience of processing, the data to be stored can be first divided into a stripe of klog2(q) bits in bit order as a sub-data segment to be processed, and each stripe, that is, each sub-data segment to be processed is divided into k groups, each group of log2(q) bits. At the same time, when storing the data to be stored, each sub-data segment to be processed and the check data of the sub-data segment to be processed can be stored one by one until all the sub-data segments to be processed are stored.

[0188] Optionally, when storing the sub-data segment to be stored and the check data of the sub-data segment to be stored, the length of each storage unit can be determined according to the encoding code length n and the dimension k, that is, how many storage nodes are needed to store the data to be stored, and at the same time, how many storage nodes among these storage nodes are data nodes and how many storage nodes are check nodes, so as to obtain each storage node corresponding to the sub-data segment to be stored. The storage unit includes a data node, a local check node and a global check node.

[0189] For example, continue to refer to Figure 4As shown, when storing a sub-data segment to be stored and the verification data of the sub-data segment to be stored, it can be determined that 16 storage nodes are required to store the sub-data segment to be stored based on the encoding code length n and the dimension k. At the same time, it is determined that 10 storage nodes among the 16 storage nodes are data nodes, 6 storage nodes are verification nodes, and specifically, there are 4 local verification nodes and 2 global verification nodes.

[0190] S1002: Determine the storage unit to which each element in the check data belongs according to the generation matrix.

[0191] Optionally, the generator matrix M includes a local check column and a global check column. When the check data y is calculated from the local check column of M, the bit is called the local check bit, and the other bits are called global check bits. Therefore, the check data includes two types: local check data and global check data. Based on this, when storing the check data, the local check data generated by the local check column in the generator matrix and the global check data generated by the global check column can be determined according to the generator matrix, so as to determine the storage unit to which each element in the check data belongs, so as to store the local check data in the local check node and the global check data in the global check node. Among them, each element in the check data corresponds one-to-one to the sub-data segment to be stored.

[0192] S1003. Store the sub-data segment to be stored and the verification data in each storage unit according to the length of the storage unit and the storage unit to which each element belongs.

[0193] Optionally, after determining the length of the storage unit and the storage unit to which each element in the verification data belongs, the sub-data segment to be stored and the verification data may be stored in corresponding nodes in each storage unit, respectively.

[0194] For example, continue to refer to Figure 4 As shown, klog2(q) bits of sub-data segments to be stored are written into k data nodes D1-D10 of the storage system, and (nk)log2(q) bits of verification data of the sub-data segments to be stored are written into nk verification nodes L1-L4 and P1-P2.

[0195] It can be understood that after the above embodiments of the present application are executed, the data to be stored can be redundantly encoded, and the data to be stored and the verification data can be stored in multiple storage nodes. Figure 4As shown, after the above-mentioned embodiments of the present application are executed, assuming that the data of the current stripe is d1-d10, local check data l1-l4 can be obtained, and global check data p1-p2 can be obtained, and d1-d10 are stored in data nodes D1-D10, local check data l1-l4 are stored in local check nodes L1-L4, and global check data p1-p2 are stored in global check nodes P1-P2.

[0196] It is worth noting that after the data to be stored and the verification data are stored in multiple storage nodes, when a storage node is damaged, the embodiment of the present application also provides the ability to handle the abnormal situation where the storage node is damaged and the stored data is lost, so that the data to be stored and the verification data can be restored after part of the data is lost. The details are explained below.

[0197] Based on the same inventive concept, the present application also provides a data repair method. Fig.11 A flow chart of a data repair method provided in an embodiment of the present application, the data repair method is used to repair the data to be stored and the verification data stored by the above-mentioned data storage encoding method, wherein the data to be stored and the verification data are divided into multiple data groups, namely local repair groups, each data group includes at least one data to be stored and at least one verification data, each data group is stored in multiple storage nodes, and the data repair method is used to repair the storage node in any local repair group that is damaged, referring to Fig.11 As shown, the method includes:

[0198] S1101, determining a node to be repaired among multiple storage nodes and a target data group where the node to be repaired is located.

[0199] The storage node may include a data node, a local check node, and a global check node. The node to be repaired is a node that is currently damaged, and may be a data node, a local check node, or a global check node. The target data group where the node to be repaired is located is the local repair group where the node that is currently damaged is located.

[0200] For example, continue to refer to Figure 4 As shown, assuming that data node D1 is damaged, it can be determined that the node to be repaired is node D1, and the target data group where the node to be repaired is located is determined to be the local repair group 301.

[0201] S1102. Determine multiple target storage nodes corresponding to the target data group.

[0202] The target storage nodes are other storage nodes in the target data group.

[0203] For example, continue to refer to Figure 4 As shown, when the node to be repaired is a D1 node, the target storage node may be a D2 node, a D3 node, and an L1 node.

[0204] S1103: Take out r target storage nodes from the multiple target storage nodes, and perform XOR processing on the data of the r target storage nodes to repair the node to be repaired.

[0205] Optionally, after determining the multiple target storage nodes corresponding to the target data group, r target storage nodes may be taken out from the multiple target storage nodes, where r is the above-mentioned locality parameter.

[0206] Optionally, after obtaining r target storage nodes, data in the r target storage nodes may be subjected to bitwise XOR processing to repair the node to be repaired, and the processed result may be obtained as the data stored in the node to be repaired.

[0207] For example, continue to refer to Figure 4 As shown, when the node to be repaired is the D1 node, the target storage node can be the D2 node, the D3 node and the L1 node. The locality parameter at this time is 3, that is, the data in the D2 node, the D3 node and the L1 node are processed bitwise XOR to obtain the data in the node D1 to be repaired.

[0208] That is to say, after redundantly storing the data to be stored by the data storage encoding method provided by the above embodiment of the present application, when a storage node of a local repair group is damaged, it is only necessary to determine the local repair group where the damaged storage node is located and the locality parameter during storage, and then the damaged storage node can be repaired by the data repair method provided by this embodiment. This greatly improves the availability and data recovery capability of the distributed storage system and optimizes the performance of the distributed storage system.

[0209] It is worth noting that the data repair method provided in the above-mentioned embodiments of the present application can not only handle the situation where a single storage node is damaged, but also can handle the abnormal situation where multiple storage nodes are damaged, resulting in the loss of stored data. Thus, after multiple storage nodes are damaged, the stored data and verification data can be restored, which is described in detail below.

[0210] In one possible implementation, Fig.12 Another flowchart of the data repair method provided in the embodiment of the present application is shown in FIG. Fig.12 As shown, the method also includes:

[0211] S1201, determining a plurality of nodes to be repaired among a plurality of storage nodes and a target data group where each node to be repaired is located.

[0212] The storage node may include a data node, a local check node, and a global check node. The node to be repaired is a node that is currently damaged, and may be a data node, a local check node, or a global check node. The target data group where the node to be repaired is located is the local repair group where the node that is currently damaged is located.

[0213] For example, continue to refer to Figure 4 As shown, assuming that data node D1, local check node L1, data node D5, data node D8, local check node L3, and data node D10 are damaged, it can be determined that the nodes to be repaired are data node D1, local check node L1, data node D5, data node D8, local check node L3, and data node D10, and the target data groups where the nodes to be repaired are located are determined to be local repair groups 301-local repair groups 304.

[0214] S1202: If each node to be repaired and the target data group where each node to be repaired is located meet preset conditions, k storage nodes are taken out from multiple storage nodes, and the data in the k storage nodes are decoded by using a check matrix to obtain a data decoding result.

[0215] Optionally, each node to be repaired and the target data group where each node to be repaired is located are judged. If damage occurs in each local repair group at this time, k storage nodes can be taken out from multiple storage nodes, and the k storage nodes are decoded by the check matrix used when storing data to obtain data decoding results. Wherein, k is the above-mentioned dimension.

[0216] Optionally, the preset condition may be that at least one storage node in each local repair group is damaged, for example, one storage node in each local repair group is damaged first, and two storage nodes in any local repair group are damaged.

[0217] For example, continue to refer to Figure 4 As shown, after determining that the nodes to be repaired are data node D1, local check node L1, data node D5, data node D8, local check node L3, and data node D10, and determining that the target data group where each node to be repaired is located is local repair group 301-local repair group 304, 8 storage nodes can be taken out from the remaining storage nodes, and the 8 storage nodes can be decoded by the check matrix used when storing data to obtain the data decoding result.

[0218] That is to say, after redundantly storing the data to be stored by the data storage encoding method provided by the above embodiment of the present application, when multiple storage nodes are damaged, the local repair groups where the multiple storage nodes are located can be judged. If there are damaged storage nodes in each local repair group, and only the local repair group where the damaged storage node is located and the dimension when stored need to be determined, the stored data can be decoded by the data decoding method provided by this embodiment. This greatly improves the availability and data recovery capability of the distributed storage system and optimizes the performance of the distributed storage system.

[0219] Based on the same inventive concept, an embodiment of the present application also provides a data storage device corresponding to the data storage encoding method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned data storage encoding method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0220] Based on the same inventive concept, an embodiment of the present application provides a data storage encoding device, which is used for:

[0221] Obtaining encoding parameters of the extremely large repairable code, the encoding parameters including: encoding code length range;

[0222] Determine the encoding code length, finite field parameter and storage parameter of the extremely large repairable code according to the encoding code length range;

[0223] Determine the check matrix of the maximally repairable code based on finite field parameters and finite projective geometry;

[0224] Generate verification data of the data to be stored according to the verification matrix and the data to be stored;

[0225] According to the encoding code length and storage parameters, the data to be stored and the verification data are stored.

[0226] In a possible implementation, the data storage encoding device is specifically used for:

[0227] Determine the finite field parameter and the encoding code length of the extremely large repairable code according to the encoding code length range and the preset linear relationship between the encoding code length and the finite field parameter, wherein the finite field parameter is a power of 2, and the linear relationship between the encoding code length and the finite field parameter is: q≤n≤4(q+1) / 3 and n is a multiple of 4, wherein n is the encoding code length and q is the finite field parameter;

[0228] According to the finite field parameter of the extremely large repairable code and the encoding code length, the storage parameter of the extremely large repairable code is calculated.

[0229] In a possible implementation, the data storage encoding device is specifically used for:

[0230] Determine the set of projection points according to the finite field parameters and the finite projective geometry;

[0231] According to the points in the projection point set, the check matrix of the maximal repairable code is determined.

[0232] In a possible implementation, the data storage encoding device is specifically used for:

[0233] Correspond the points in the projection point set to two-dimensional vectors in the two-dimensional linear space, and obtain a system of linear equations;

[0234] The linear equations are solved, and the solutions of the linear equations are substituted into the initial check matrix to obtain the check matrix of the maximally repairable code.

[0235] In one possible implementation, the linear system is a i =λ i，1 b i -λ i，2 c i , where a i , b i , c i is a projection point in the projection point set, λ i，1 ,λ i，2 is the unknown number that needs to be solved;

[0236] In a possible implementation, the data storage encoding device is specifically used for:

[0237] If the solution of the linear equation system is (c i，1 , c i，2 ),make and will Substitute into the initial check matrix to obtain the check matrix.

[0238] In one possible implementation, the first set of values ​​of the extremely large repairable code includes: code length Dimensionality The locality parameter r = 3, the finite field parameter q = 8; the second set of values ​​of the extremely repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 16;

[0239] The third set of values ​​for extremely large repairable codes includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 32;

[0240] The fourth set of values ​​for extremely large repairable codes includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 64;

[0241] The fifth set of values ​​for extremely large repairable codes includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 128;

[0242] The sixth set of values ​​for extremely large repairable codes includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 256;

[0243] The seventh set of values ​​for extremely large repairable codes includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 512;

[0244] The second set of values ​​for extremely large repairable codes include: code length Dimensionality The locality parameter r=3, the finite field parameter q=1024, where Is a positive integer.

[0245] Based on the same inventive concept, a data repair device corresponding to the data repair method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned data repair method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0246] Another embodiment of the present application provides a data repair device, which is used to repair the data to be stored and the verification data stored by the above data storage encoding method, wherein the data to be stored and the verification data are divided into multiple data groups, each data group includes at least one data to be stored and at least one verification data, and each data group is stored in multiple storage nodes respectively. The data repair device is used to:

[0247] Determine a node to be repaired among multiple storage nodes and a target data group where the node to be repaired is located;

[0248] Determine multiple target storage nodes corresponding to the target data group;

[0249] R target storage nodes are taken out from the multiple target storage nodes, and data of the r target storage nodes are XOR-processed to repair the node to be repaired.

[0250] In a possible implementation, the data repair device is specifically used for:

[0251] Determine a plurality of nodes to be repaired among a plurality of storage nodes and a target data group where each of the nodes to be repaired is located;

[0252] If each node to be repaired and the target data group where each node to be repaired is located meet the preset conditions, k storage nodes are taken out from multiple storage nodes, and the data in the k storage nodes are decoded by the check matrix to obtain a data decoding result.

[0253] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0254] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned data storage encoding method or data repair method are executed.

[0255] An embodiment of the present application also provides a computer program product, which includes a computer program code. When the computer program code is executed by a computing device, the computing device executes the steps of the above-mentioned data storage encoding method or data repair method.

[0256] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0257] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the technical solution that contributes to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.

[0258] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A data storage encoding method, It is characterized in that include: Obtaining encoding parameters of the extremely large repairable code, the encoding parameters including: an encoding code length range; Determining the encoding code length, finite field parameters, and storage parameters of the extremely large repairable code according to the encoding code length range; Determining a check matrix of the maximal repairable code according to the finite field parameters and the finite projective geometry; Generating verification data of the data to be stored according to the verification matrix and the data to be stored; According to the encoding code length and the storage parameter, the data to be stored and the verification data are stored.

2. The data storage encoding method according to claim 1, It is characterized in that The step of determining the encoding code length, finite field parameters, and storage parameters of the extremely large repairable code according to the encoding code length range includes: Determine the finite field parameter and the encoding code length of the extremely large repairable code according to the encoding code length range and the preset linear relationship between the encoding code length and the finite field parameter, wherein the finite field parameter is a power of 2, and the linear relationship between the encoding code length and the finite field parameter is: q≤n≤4(q+1) / 3, and n is a multiple of 4, wherein n is the encoding code length, and q is the finite field parameter; According to the finite field parameter of the extremely large repairable code and the encoding code length, the storage parameter of the extremely large repairable code is calculated.

3. The data storage encoding method according to claim 1, It is characterized in that The step of determining the check matrix of the maximally repairable code according to the finite field parameters and the finite projective geometry comprises: According to the finite field parameters and the finite projective geometry PG(1,q), determine the set of projection points; A check matrix of the maximal repairable code is determined according to the points in the projection point set.

4. The data storage encoding method according to claim 3, It is characterized in that The step of determining the check matrix of the maximal repairable code according to the points in the projection point set comprises: Corresponding the points in the projection point set to two-dimensional vectors in a two-dimensional linear space to obtain a linear equation system; The linear equations are solved, and the solutions of the linear equations are substituted into an initial check matrix to obtain the check matrix of the maximally repairable code.

5. The data storage encoding method according to claim 4, It is characterized in that The linear equation system is a i =λ i, 1 b i -λ i,2 c i , where a i ,b i ,c i is a projection point in the projection point set, λ i,1 ,λ i,2 is the unknown number that needs to be solved; Substituting the solution of the linear equations into an initial check matrix to obtain the check matrix of the maximally repairable code includes: If the solution of the linear equations is (c i,1 ,c i,2 ),make and will Substitute into the initial check matrix to obtain the check matrix.

6. The data storage encoding method according to any one of claims 1 to 5, It is characterized in that The first set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 8; The second set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 16; The third set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 32; The fourth group of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 64; The fifth set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 128; The sixth set of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 256; The seventh group of values ​​of the extremely large repairable code includes: code length Dimensionality Locality parameter r = 3, finite field parameter q = 512; The second set of values ​​of the extremely large repairable code includes: code length Dimensionality The locality parameter r=3, the finite field parameter q=1024, where Is a positive integer.

7. A data repair method, It is characterized in that Used to repair the data to be stored and the verification data stored in any one of claims 1 to 6 above, wherein the data to be stored and the verification data are divided into a plurality of data groups, each of the data groups includes at least one data to be stored and at least one verification data, and each of the data groups is stored in a plurality of storage nodes, the method comprising: Determine a node to be repaired among the multiple storage nodes and a target data group where the node to be repaired is located; Determine a plurality of target storage nodes corresponding to the target data group; R target storage nodes are taken out from the multiple target storage nodes, and data of the r target storage nodes are XOR-processed to repair the node to be repaired.

8. The data repair method according to claim 7, It is characterized in that The method further comprises: Determine a plurality of nodes to be repaired among a plurality of storage nodes and a target data group where each of the nodes to be repaired is located; If each node to be repaired and the target data group where each node to be repaired is located meet the preset conditions, k storage nodes are taken out from the multiple storage nodes, and the data in the k storage nodes are decoded by using a check matrix to obtain a data decoding result.

9. A data storage encoding device, It is characterized in that The data storage encoding device is used for: Obtaining encoding parameters of the extremely large repairable code, the encoding parameters including: an encoding code length range; Determining the encoding code length, finite field parameters, and storage parameters of the extremely large repairable code according to the encoding code length range; Determining a check matrix of the maximal repairable code according to the finite field parameters and the finite projective geometry; Generating verification data of the data to be stored according to the verification matrix and the data to be stored; According to the encoding code length and the storage parameter, the data to be stored and the verification data are stored.

10. A data repair device, It is characterized in that Used to repair the data to be stored and the verification data stored in any one of claims 1 to 6 above, wherein the data to be stored and the verification data are divided into a plurality of data groups, each of the data groups includes at least one data to be stored and at least one verification data, and each of the data groups is stored in a plurality of storage nodes, the method comprising: Determine a node to be repaired among the multiple storage nodes and a target data group where the node to be repaired is located; Determine a plurality of target storage nodes corresponding to the target data group; R target storage nodes are taken out from the multiple target storage nodes, and data of the r target storage nodes are XOR-processed to repair the node to be repaired.

11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are executed.

12. A computer program product, It is characterized in that The computer program product comprises computer program codes. When the computer program codes are executed by a computing device, the computing device executes the steps of the method according to any one of claims 1 to 8.

Images