Rack sensing distributed storage system based on local repair code and repair method thereof

Abstract

The invention discloses a rack sensing distributed storage system based on a local repair code and a repair method thereof, and the system comprises a plurality of racks, each rack comprises a plurality of storage nodes; the coding module is used for coding the original data and distributing the original data to each storage node; and the repairing module is used for executing a corresponding repairing strategy according to the number of the fault nodes. It can be seen that by designing a rack sensing coding scheme on a small finite field, the calculation complexity caused by finite field operation can be reduced, the flexibility of system parameters of the coding scheme can be improved, and compared with a traditional rack sensing array code coding scheme, the cross-rack repairing bandwidth can be smaller when fault nodes are repaired.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a rack-aware distributed storage system based on local repair codes and a repair method thereof. Background Art

[0002] In distributed storage systems, the reliability of disk drives is far lower than theoretical expectations[1], resulting in frequent failures and damages of storage nodes[2]. The reliability of distributed storage systems has been improved by introducing redundancy strategies and fault-tolerant mechanisms. Traditional redundancy strategies include replication strategies and erasure coding schemes.

[0003] The replication strategy directly stores copies of the original data in multiple storage nodes, and can repair faulty nodes by directly reading data from other normal nodes. The replication strategy has a large storage overhead. In actual data centers, more than 55% of the economic expenses come from buildings, power systems, and equipment expenses directly related to storage [3]. When the scale of data storage increases, the disadvantage of high storage overhead becomes more obvious.

[0004] Erasure coding schemes can significantly reduce storage overhead. Usually, the system maximum distance separable code (MDS code) or Reed-Solomon code is used as the coding strategy for distributed storage systems. Storage systems based on MDS codes have the advantages of high reliability and low storage overhead. However, storage systems based on MDS codes have high repair overhead. The repair process will generate large bandwidth occupancy and computing requirements, which can easily cause system response delays or even unavailability. For example, more than 20% of the network congestion of Facebook's cloud storage service is caused by storage data based on MDS codes [4].

[0005] Dimakis et al. [5] proposed a distributed storage system based on regeneration codes. In this new distributed storage system, when a storage node fails, the regeneration code mechanism can repair the failed node by downloading part of the data information on some other storage nodes. The regeneration code technology reduces the communication bandwidth generated by the repair of the failed node by designing a repair scheme of erasure code. However, the traditional repair scheme does not consider the relative position of the nodes in the distributed system. The storage nodes of the distributed storage system are generally organized in racks. Obviously, the communication overhead of the nodes within the rack is much smaller than the communication overhead between nodes. Hu et al. [6][7] proposed a rack-aware storage model, assuming that the nodes in the system are divided into groups (racks) of the same size, and the communication within the rack does not occupy the network bandwidth of the system. Chen et al. [8] designed a rack-aware storage system with optimal repair bandwidth. When repairing a failed node, it is necessary to download part of the data stored in other normal nodes in the current rack and part of the data in some other racks. Cai et al. [9] designed a rack-aware storage coding scheme by adding redundancy in the rack, which only needs to download part of the data stored in other normal nodes in the rack when repairing one or a small number of failed nodes.

[0006] However, the rack-aware storage coding scheme designed by Chen et al. [8] still requires cross-rack communication when repairing a small number of faulty nodes. The rack-aware storage coding scheme designed by Cai et al. [9] is based on finite field operations and requires that the number of racks multiplied by the number of nodes contained in each rack is less than or equal to the size of the finite field. In other words, Cai et al.'s coding scheme requires implementation on a larger finite field, which makes the encoding and repair scheme have higher computational complexity. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a rack-aware distributed storage system based on local repair codes and a repair method thereof, which can reduce the computational complexity caused by finite field operations and improve the flexibility of system parameters of the coding scheme by designing a rack-aware coding scheme on a smaller finite field. Compared with the traditional rack-aware array code coding scheme, it can have a smaller cross-rack repair bandwidth when repairing a faulty node.

[0008] In order to solve the above technical problems, the first aspect of the present invention discloses a rack-aware distributed storage system based on local repair codes, the system comprising:

[0009] A plurality of racks, each of the racks comprising a plurality of storage nodes;

[0010] An encoding module, which is used to encode the original data and distribute it to each storage node;

[0011] The repair module is used to execute a corresponding repair strategy according to the number of faulty nodes.

[0012] As an optional implementation manner, in the first aspect of the present invention, encoding the original data and distributing it to each storage node includes:

[0013] According to the original data matrix, the encoded data matrix is ​​calculated to obtain the encoding matrix;

[0014] The elements in the encoded data matrix are stored in the corresponding storage nodes according to the preset rules;

[0015] Wherein, the encoding module adopts a system code encoding strategy.

[0016] As an optional implementation, in the first aspect of the present invention, storing the elements in the encoded data matrix in corresponding storage nodes according to a preset rule includes:

[0017] The position information of each element in the encoding matrix is ​​obtained, and each element is stored in a corresponding storage node in a corresponding rack according to the corresponding position information.

[0018] As an optional implementation, in the first aspect of the present invention, a monitoring module is further included, wherein the monitoring module is used to monitor the status of the storage node and notify the repair module when a fault is detected.

[0019] A second aspect of the present invention discloses a rack-aware distributed storage system repair method, which is used to repair the rack-aware distributed storage system based on the local repair code as claimed in the claim, and is characterized by comprising the following steps:

[0020] Detect the faulty nodes of the rack and obtain the number of faulty nodes in each rack;

[0021] Select the corresponding repair strategy based on the number of faulty nodes;

[0022] Execute the repair strategy to restore the data of the failed node.

[0023] As an optional implementation, in the second aspect of the present invention, the repair strategy includes a local repair strategy and a cross-rack repair strategy.

[0024] As an optional implementation, in the second aspect of the present invention, when the number of failed nodes is less than a preset value, a local repair strategy is executed;

[0025] When the number of failed nodes is equal to or greater than the preset value, the cross-rack repair strategy is executed.

[0026] As an optional implementation, in the second aspect of the present invention, the local repair strategy includes:

[0027] Randomly download a preset number of normal node data from the faulty rack;

[0028] Using the Lagrange interpolation formula, calculate the expression of the first polynomial based on the downloaded data;

[0029] The faulty node data is restored according to the calculated first polynomial.

[0030] As an optional implementation, in the second aspect of the present invention, the cross-rack repair strategy includes:

[0031] Download the data of the first preset number of normal nodes of any number of normal racks;

[0032] Using the downloaded data, the data of the faulty rack is calculated through interpolation and mapping techniques.

[0033] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0034] In the embodiment of the present invention, when repairing one or a small number of faulty nodes, it is only necessary to download the data stored in other normal nodes in the rack, and the values ​​of the number of racks and the rack size relative to the size of the finite field are increased, which solves the problem that the existing rack-aware storage coding scheme requires that the number of racks multiplied by the number of nodes contained in each rack is less than or equal to the size of the finite field. The rack-aware storage coding scheme in the embodiment of the present invention can be applicable to the case where the number of racks is less than or equal to the size of the finite field and the rack size is less than or equal to the size of the finite field; under the same system parameters, a coding based on a smaller finite field can be designed, thereby reducing the complexity of the system's encoding and repair scheme; in the coding scheme based on the same finite field operation, the coding scheme proposed in the embodiment of the present invention has more flexible system parameters of the number of racks and the size of the rack. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 It is a structural schematic diagram of a rack-aware distributed storage system based on local repair codes disclosed in an embodiment of the present invention;

[0037] Figure 2 is a structural diagram of a rack-aware storage model disclosed in an embodiment of the present invention;

[0038] Figure 3It is a flowchart of a rack-aware distributed storage system repair method disclosed in an embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of a local repair strategy disclosed in an embodiment of the present invention;

[0040] Figure 5 It is a schematic diagram of the cross-rack repair strategy disclosed in the embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0043] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0044] The present invention discloses a rack-aware distributed storage system based on local repair codes and a repair method thereof. When repairing one or a small number of faulty nodes, only the data stored in other normal nodes in the rack need to be downloaded, and the values ​​of the number of racks and the rack size relative to the size of a finite field are increased, solving the problem that the existing rack-aware storage coding scheme requires that the number of racks multiplied by the number of nodes contained in each rack is less than or equal to the size of the finite field. The rack-aware storage coding scheme in the embodiment of the present invention can be applied to the case where the number of racks is less than or equal to the size of the finite field and the rack size is less than or equal to the size of the finite field; under the same system parameters, a coding based on a smaller finite field can be designed, thereby reducing the complexity of the system's encoding and decoding and repair schemes; in the coding scheme based on the same finite field operation, the coding scheme proposed in the embodiment of the present invention has more flexible system parameters of the number of racks and the size of the rack. The following are detailed descriptions.

[0045] Embodiment 1

[0046] See also Figure 1 , Figure 1 Schematic diagram of a rack-aware distributed storage system based on local repair codes disclosed in an embodiment of the present invention. Figure 1 As shown, the rack-aware distributed storage system based on local repair codes includes:

[0047] A plurality of racks 100, each of which comprises a plurality of storage nodes;

[0048] An encoding module 200, which is used to encode the original data and distribute it to each storage node;

[0049] The repair module 300 is used to execute a corresponding repair strategy according to the number of faulty nodes.

[0050] In an optional implementation, encoding the original data and distributing it to each storage node includes:

[0051] Calculate the original data matrix and generate the encoded data matrix to obtain the encoding matrix;

[0052] The elements in the encoded data matrix are stored in the corresponding storage nodes according to the preset rules;

[0053] Wherein, the encoding module adopts a system code encoding strategy.

[0054] In an optional implementation, storing the elements in the encoded data matrix in corresponding storage nodes according to a preset rule includes:

[0055] The position information of each element in the encoding matrix is ​​obtained, and each element is stored in a corresponding storage node in a corresponding rack according to the corresponding position information.

[0056] In an optional implementation, a monitoring module is further included, wherein the monitoring module is used to monitor the status of the storage node and notify the repair module when a failure is detected.

[0057] In the embodiment of the present invention, refer to Figure 2 Assume that the storage system contains a total of l racks, each rack contains r+δ-1 nodes. The first r nodes in the first t racks are used to store the original data. Assume that the original data can be expressed as

[0058]

[0059] where x ij is a finite field elements on. Let p1,p2,…,p l is a different prime number that is relatively prime to r. The characteristic of is p. Select l different elements δ1,δ2,…,δ l , so that the finite field The algebraic extension of The number of times is p i , where i=1,2,…,l.

[0060] In finite fields Select r+δ-1 different elements β1,β2,…,β r+δ-1 . Let the matrix

[0061]

[0062] The encoding process of the original data (1) is as follows:

[0063] Step 1: Calculate matrix I from the original data matrix I b =(b ij ) 1≤i≤r；1≤j≤t The calculation formula is as follows:

[0064]

[0065] Step 2: For any 1≤i≤r+δ-1 and 1≤j≤l, the i-th node on rack j stores a The element y on ij , where y ij The calculation formula is as follows:

[0066]

[0067] Step 3: The data after encoding the original data I is the following matrix:

[0068]

[0069] Store the element in the i-th row and j-th column of the matrix in the i-th node on rack j.

[0070] This encoding scheme is a systematic code, and users can obtain the original data by accessing the first r nodes in the first t racks. Therefore, this encoding scheme has a high data access efficiency.

[0071] Embodiment 2

[0072] See also Figure 3 , Figure 3 FIG. 1 is a flow chart of a rack-aware distributed storage system repair method disclosed in an embodiment of the present invention. Figure 3 As shown, the rack-aware distributed storage system repair method includes the following steps:

[0073] 101. Detect the faulty nodes of the racks and obtain the number of faulty nodes of each rack;

[0074] 102. Select a corresponding repair strategy based on the number of faulty nodes;

[0075] 103. Execute the repair strategy to restore the data of the failed node.

[0076] In an optional embodiment, the repair strategy includes a local repair strategy and a cross-rack repair strategy.

[0077] In an optional embodiment, when the number of failed nodes is less than a preset value, a local repair strategy is executed; when the number of failed nodes is equal to or greater than a preset value, a cross-rack repair strategy is executed.

[0078] In an optional embodiment, the local repair strategy includes:

[0079] Randomly download a preset number of normal node data from the faulty rack;

[0080] Using the Lagrange interpolation formula, calculate the expression of the first polynomial based on the downloaded data;

[0081] The faulty node data is restored according to the calculated first polynomial.

[0082] In the embodiment of the present invention, when the number of failed nodes in a rack is less than δ, local repair can be performed by downloading data of other r normal nodes in the rack.

[0083] Assume that the first u nodes in the first rack fail, where u < δ. For any integer i that satisfies 1 ≤ i ≤ r + δ - 1, we can get from formula (2)

[0084]

[0085] make

[0086] and

[0087] Then for any 1≤i≤r+δ-1,

[0088] y i1 =f(β i ).

[0089] Repair process Figure 4 shown.

[0090] The specific repair plan is as follows:

[0091] Step 1: Randomly download data from r normal nodes in the faulty rack

[0092] Step 2: According to the Lagrange interpolation formula, there are r value points Evaluate the expression of the polynomial f(x).

[0093] Step 3: Based on the polynomial f(x) obtained in step 2, i1 =f(β i ), 1≤i≤r+δ-1 can calculate the data y of the faulty node 11 ,...,y u1 .

[0094] In an optional embodiment, the cross-rack repair strategy includes:

[0095] Download the data of the first preset number of normal nodes of any number of normal racks;

[0096] Using the downloaded data, the data of the faulty rack is calculated through interpolation and mapping techniques.

[0097] In the embodiment of the present invention, when the number of failed nodes in a rack is greater than or equal to δ, the entire rack is considered to be failed. Repairing the entire failed rack requires access to data stored in other normal racks, and such repair will generate cross-rack repair bandwidth.

[0098] Assume that rack 1 fails. The first r nodes of any other t racks are used to repair the data of the first rack, and the resulting cross-rack repair bandwidth can reach the minimum value proved in theory.

[0099] First, let's explain some symbols. For any integers i, j satisfying 1≤i≤r and 1≤j≤t+1, let

[0100]

[0101] but

[0102] y ij =f j (β i ). (4)

[0103] set up is a finite field The algebraic extension domain with the expansion number r is set Make Define a arrive The bijection of:

[0104]

[0105] For any integer j that satisfies 1≤j≤t+1, let

[0106]

[0107] According to f hj The definition of

[0108] in

[0109] The faulty rack 1 can be restored by any other t racks. The following describes how to restore the faulty rack 1 by using rack 2, rack 3, ..., rack t+1. The repair process is as follows: Figure 5 shown.

[0110] The specific repair process is as follows:

[0111] Step 1: For a normal rack j, where 2≤j≤t+1, download the data y stored by the first r nodes 1j ,y 2j ,…,y rj According to the above formula (4), using the Lagrange interpolation formula, the r value points (β i1 ,y i11 ), Evaluate the polynomial f j (x), we get f j The coefficient f of (x) 1j ,f 2j ,…,f rj .

[0112] Step 2: Use the coefficient f obtained in the first step 1j ,f2j ,…,f rj , 2≤j≤t+1, calculate (f 1j ,f 2j ,…,f rj ) in the mapping The image below The calculation formula is shown in formula (5).

[0113] Step 3: Use the Lagrange interpolation formula to evaluate the point Get the polynomial expression.

[0114] Step 4: Use the polynomial obtained in step 3 Calculate the value of the polynomial at point δ1

[0115] Step 5: Calculate the value obtained in step 4 In the mapping The image below (f 11 ,f 21 ,…,f r1 ), we get the polynomial

[0116] Step 6: Calculate the polynomial obtained in step 5 At the evaluation points β1,β2,…,β r+δ-1 The value of y i1 =f1(β i ). Get all the data y stored in rack 1 11 ,y 21 , …, y r+δ-1,1 .

[0117] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0118] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0119] Finally, it should be noted that the rack-aware distributed storage system based on local repair codes and its repair method disclosed in the embodiments of the present invention are only the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that it is still possible to modify the technical solutions recorded in the aforementioned embodiments, or to make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

[0120] The references cited in the background technology are as follows:

[0121] [1]B.Schroeder,G.A.Gibson.Disk failures in the real world:What doesan MTTFof 1,000,000hours mean to you?.In Proceedings of the USENIX ConferenceonFile and Storage Technologies,2007.

[0122] [2]K.Rashmi,N.B.Shah,D.Gu,H.Kuang,D.Borthakur,K.Ramchandran.Asolutionto the network challenges of data recovery in erasure-codeddistributedstorage systems:A study on the facebook warehouse cluster.InProceedings of the

[0123] 5th USENIX Workshop on Hot Topics in Storage and File Systems(HotStorage

[0124] 13),2013.

[0125] [3]C.Huang,M.Chen,J.Li.Pyramid Codes:Flexible Schemes to Trade SpaceforAccess Efficiency in Reliable Data Storage Systems.In Proceedings of the6thIEEE International Symposium on Network Computing and Applications,NCA

[0126] 2007(IEEE),2007:79-86.

[0127] [4]D.S.Papailiopoulos,A.G.Dimakis.Locally repairable codes.IEEETransactionson Information Theory,2014,60(10):5843-5855.

[0128] [5]A.G.Dimakis,P.B.Godfrey,Y.Wu,M.J.Wainwright,K.Ramchandran.Networkcoding for distributed storage systems.IEEE Transactionson Information Theory,

[0129] 2010,56(9):4539–4551.

[0130] [6]Y.Hu,P.P.Lee,and X.Zhang,“Double regenerating codes forhierarchical datacenters,”in 2016IEEE International Symposium on InformationTheory(ISIT).

[0131] IEEE,2016,pp.245–249.

[0132] [7]Y.Hu,X.Li,M.Zhang,P.P.Lee,X.Zhang,P.Zhou,and D.Feng,“Optimalrepairlayering for erasure-coded data centers:From theory to practice,”ACMTransactions on Storage(TOS),vol.13,no.4,pp.1–24,2017.

[0133] [8]Z.Chen and A.Barg,“Explicit constructions of msr codes forclustered distributedstorage:The rack-aware storage model,”IEEE Transactionson Information Theory,

[0134] vol.66,no.2,pp.886–899,2019.

[0135] [9]H.Cai,Y.Miao,M.Schwartz,and X.Tang,“Repairing schemes for tamo-bargcodes,”arXiv preprint arXiv:2312.12803,2023.

Claims

1. A rack-aware distributed storage system based on local repair codes, characterized in that: The system comprises: A plurality of racks, each of the racks comprising a plurality of storage nodes; An encoding module, which is used to encode the original data and distribute it to each storage node; The repair module is used to execute a corresponding repair strategy according to the number of faulty nodes.

2. The rack-aware distributed storage system based on local repair codes according to claim 1, characterized in that: The encoding of the original data and distributing it to each storage node includes: According to the original data matrix, the encoded data matrix is ​​calculated to obtain the encoding matrix; The elements in the encoded data matrix are stored in the corresponding storage nodes according to the preset rules; Wherein, the encoding module adopts a system code encoding strategy.

3. The rack-aware distributed storage system based on local repair codes according to claim 2, characterized in that: The step of storing the elements in the encoded data matrix in corresponding storage nodes according to a preset rule includes: The position information of each element in the encoding matrix is ​​obtained, and each element is stored in a corresponding storage node in a corresponding rack according to the corresponding position information.

4. The rack-aware distributed storage system based on local repair codes according to claim 1, characterized in that: It also includes a monitoring module, which is used to monitor the status of the storage node and notify the repair module when a failure is detected.

5. A rack-aware distributed storage system repair method, used to repair the rack-aware distributed storage system based on local repair code according to any one of claims 1 to 4, characterized in that: The following steps are involved: Detect the faulty nodes of the rack and obtain the number of faulty nodes in each rack; Select the corresponding repair strategy based on the number of faulty nodes; Execute the repair strategy to restore the data of the failed node.

6. The rack-aware distributed storage system repair method according to claim 5, characterized in that: The repair strategy includes a local repair strategy and a cross-rack repair strategy.

7. The rack-aware distributed storage system repair method according to claim 6, characterized in that: When the number of failed nodes is less than the preset value, the local repair strategy is executed; When the number of failed nodes is equal to or greater than the preset value, the cross-rack repair strategy is executed.

8. The rack-aware distributed storage system repair method according to claim 6, characterized in that: The local repair strategy includes: Randomly download a preset number of normal node data from the faulty rack; Using the Lagrange interpolation formula, calculate the expression of the first polynomial based on the downloaded data; The faulty node data is restored according to the calculated first polynomial.

9. The rack-aware distributed storage system repair method according to claim 6, characterized in that: The cross-rack repair strategy includes: Download the data of the first preset number of normal nodes of any number of normal racks; Using the downloaded data, the data of the faulty rack is calculated through interpolation and mapping techniques.

Images