CHANGING ERROR CORRECTION CONFIGURATIONS

Abstract

In some examples, a system groups a multitude of blocks of coded data into first segments, where each first segment comprises several blocks from the multitude of blocks, and the coded data is based on the application of a deletion correction coding to input data. The system stores the first segments in corresponding error domains of a first set of error domains in a first deletion correction configuration, where the first deletion correction configuration allows error recovery in response to lost coded data in a specified number of error domains. The system transitions from the first deletion correction configuration to a second deletion correction configuration by reorganizing the first segments into second segments, where each second segment comprises a different number of blocks of coded data than a first segment.

Description

background

[0001] A storage system can be used to store data. To prevent data loss, the data can be distributed across multiple storage devices within the system, and redundant data can be added so that the original data can be recovered if the storage devices fail (either partially or completely). The redundant data can be in the form of a copy of the original data or in the form of encoded data created by applying error-correcting encoding to the original data.

[0002] US 2021 / 0132851A1 relates to the recovery of data from a data block stored across a geographically distributed data storage system, where the geographically distributed data storage system uses erasure coding technology.

[0003] CHO, Wan Hee; DATTA, Anwitaman: Elastic erasure coding for adaptive redundancy. In: Proceedings / 2016 IEEE 36 th International Conference on Distributed Computing Systems Workshops, 27-30 June 2016, pp. 176-183 refers to an elastic deletion coding for adaptive redundancy. Brief description

[0004] A non-transitory, machine-readable storage medium according to claims 1 to 10, a system according to claim 11 and a method according to claims 12 to 13 is disclosed. Brief description of the drawings

[0005] Some embodiments of the present disclosure are described with reference to the following figures. Fig. is a block diagram of an arrangement comprising multiple fault domains and a flexible coded data management engine, as shown in some examples in the present disclosure. Fig. is a block diagram that shows an arrangement with a larger number of error domains, as illustrated by some examples. Fig. is a block diagram of a storage medium that stores machine-readable instructions according to some examples. Fig. This is a block diagram of a system, based on some examples. Fig. This is a flowchart of a process, based on some examples.

[0006] In the drawings, identical reference numbers denote similar, but not necessarily identical, elements. The illustrations are not necessarily to scale, and the size of some parts may be exaggerated to make the example shown clearer. Furthermore, the drawings contain examples and / or embodiments that correspond to the description; however, the description is not limited to the examples and / or embodiments shown in the drawings. Detailed description

[0007] In this disclosure, the use of the terms "a" or "the" includes the plural forms unless the context clearly indicates otherwise. Similarly, the terms "including," "comprises," or "have," when used in this disclosure, specify the presence of the elements indicated but do not exclude the presence or addition of other elements.

[0008] The term “original data” used here can refer to a version of data that is to be written to a storage system by a write operation initiated by a requester (a user, a program, or a machine).

[0009] A storage system can comprise a number of physical storage devices. A "storage system" can also be referred to as a "storage pool," "storage array," or any other term that describes the presence of multiple physical storage devices within a storage array. Examples of a storage device include any one or a combination of the following: a disk-based storage device, a solid-state drive, and so on.

[0010] Applying deletion correction coding to the original data creates internally redundant coded data that can be used to recover the original data in the event of data loss, e.g., due to the failure of a storage device (or part of a storage device), a program error, a malware attack, or for other reasons.

[0011] An example of a deletion correction code that can be used to protect data is the Reed-Solomon deletion code. To be able to recover n (n > 1) blocks of the original data if some m > 0 blocks are lost, applying the Reed-Solomon deletion code can produce coded data that has n + m blocks, where m > 0.

[0012] The n + m blocks are stored on n + m separate storage devices of the storage system. Each block of encoded data is stored on a different storage medium of the storage system. A "block," as used here, can refer to any portion of data (where the portion of data has a specific size) that is separated from a larger collection of data. If the larger collection of data is not evenly divided into n blocks, the last block can be padded with zeros.

[0013] Recovering the original data from the encrypted data is possible if at least n blocks of the encrypted data survive. If n ≥ m (n being much larger than m), the relative storage space required to add m redundant data blocks is small, and data recovery can be efficient.

[0014] In some examples, d storage devices in a storage system can be used to store encrypted data, where d > n + 2m. For example, the data blocks of the encrypted data can be distributed across the d storage devices in the storage system. Having more than n + m storage devices in the storage system allows a failover system to guarantee that it can still provide a desired level of data redundancy even if multiple storage devices fail and cannot be replaced for an extended period. The data blocks of the additional encrypted data can be distributed differently across the d storage devices, so that approximately the same total amount of data can be stored on each device.

[0015] In other examples, erasure correction codes, which differ from the Reed-Solomon erasure code, can use m blocks of redundant data to protect n blocks of original data. Other codes, such as Golay codes or Bose-Chaudhuri-Hocquenghem (BCH) codes, can also be used. In the simplest case, a block of redundant data is calculated by performing an exclusive-OR operation on the n blocks of original data (or a modified version of the n blocks of original data). An n+1 code constructed in this way using exclusive-OR can be used to build RAID-4 or RAID-5 storage systems.

[0016] In general, an n + m delete correction code supports the recovery of n blocks of the original data in the event of a loss of up to m blocks.

[0017] Once a storage system is configured to support a specific erase correction code, such as a Reed-Solomon erase code, and n + m blocks are distributed across n + m block storage devices, the encoding of that data is fixed and will not change unless the data is completely copied into a new form with new redundant data. In such examples, changing the encoding of data (e.g., to distribute encoded data based on the original data across a changed number of storage devices, such as by adding storage devices to expand the storage capacity of a storage system) would require recalculating the encoded data based on the entire collection of the original data, which is computationally intensive.

[0018] In accordance with some implementations of the present disclosure, mechanisms or techniques are provided to support flexible data encoding for privacy purposes, in which the data encoding can be efficiently changed on the fly when a storage system changes, e.g., by adding or removing storage devices (so that encoded data can be distributed across a changed set of storage devices without having to completely recalculate the encoded data, and in some cases simply rearrange data and delete some data).

[0019] More generally, flexible coded data, generated by applying a deletion correction coding (e.g., Reed-Solomon deletion coding), allows the number of error domains to be changed by reorganizing blocks of coded data. This change in the number of error domains can be achieved without recalculating any coded data, or by recalculating only a small amount of coded data while reusing most of the existing blocks. The flexible coded data can be expanded to include more error domains or compressed to include fewer.

[0020] Fig. Figure 102 is a block diagram of a sample arrangement containing a collection of fault domains. A "fault domain" can refer to an entity associated with data storage, where data loss can occur at the entity (e.g., due to an entity failure), and where the fault domain is part of a collection of fault domains that enable data recovery after data loss in the fault domain. In some examples, the collection of fault domains might include a collection of storage devices in a storage system. In other examples, the collection of fault domains might include a collection of server computers or other machines, a collection of programs such as virtual machines or application programs, and so on.

[0021] In an example where the collection of failover domains 102 comprises a collection of server computers, the server computers are used to manage access to data in storage devices. Each server computer can be connected to a corresponding collection of storage devices (a single storage device or multiple storage devices), and the server computer manages access to the data in the connected collection of storage devices. In a "shared-nothing" arrangement, the collection of storage devices connected to one server computer is not shared with any other server computer.

[0022] The failure of a specific server computer within a collection of servers can lead to data loss. A deletion correction code, such as the Reed-Solomon deletion code, can enable data recovery after data loss in up to m failure domains, as in examples where n + m deletion correction codes are used in the collection of failure domains.

[0023] The collection of error domains 102 can be part of an overall system 100 (or more simply, a "system") that supports the storage of data in storage devices. The system 100 can include server computers connected to storage devices. The server computers can, for example, receive read and write requests from requesters to access data in the storage devices. Alternatively, the system 100 can consist of a storage system containing storage devices that requesters can access without passing through intermediaries such as server computers.

[0024] In one example, System 100 can initially be built from an initial number of fault domains 102 (e.g., an initial number of server computers, an initial number of storage devices, etc.). Over time, the number of fault domains in System 100 can change, either due to expansion, where one or more new fault domains are added to System 100 (e.g., to increase storage capacity), or due to shrinkage, where one or more existing fault domains are removed (e.g., due to data loss, for maintenance, to reduce storage capacity, etc.). Fig. The added or removed error domains are displayed as "Modified Error Domain(s) 104". Modified Error Domain(s) 104 can therefore represent either a newly added error domain or a removed existing error domain.

[0025] Note that there can be only one or more than one modified error domain 104 (e.g., by adding several new error domains or removing several existing error domains).

[0026] In one example, System 100 might start with a relatively small number of failure domains 102. To support m = 2 (where data recovery is possible in response to the loss of up to two failure domains 102), the number of non-redundant blocks is restricted so that n ≤ s - m, where s is the number of failure domains 102. If s is relatively small (e.g., s = 4), then the erase correction coding technique used can be relatively inefficient, since the number of redundant failure domains (e.g., 2) represents a significant portion of the total number of failure domains 102 (e.g., 4). Alternatively, to improve efficiency in the use of storage resources (e.g., server computers or storage devices), the value of m can be reduced, thereby decreasing the number of tolerable failures.

[0027] If the number of fault domains increases, e.g. by adding a new fault domain, it may be feasible to change the deletion correction coding technique to increase efficiency (e.g. by switching from a 2+2 deletion correction coding technique (n+m, where n=2 and m=2) to a 3+2 deletion correction coding technique (n+m, where n=3 and m=2) by increasing n or by increasing the number of tolerable failures (i.e. by increasing m).

[0028] Conversely, the number of fault domains can be reduced, for example, due to hardware failures or other problems. For instance, a hardware failure or other issue at a remote location where regular maintenance is difficult might require System 100 to operate for an extended period without at least one fault domain. If the data is initially stored in n + m fault domains, the loss of one or more fault domains might make it desirable to reconfigure System 100 to store n' + m blocks where n' < n. This reconfiguration might allow System 100 to maintain the same fault rate m without data loss, at the cost of less efficient storage utilization.

[0029] In accordance with some implementations of the present disclosure, System 100 includes a flexible coded data management engine 106 that supports reconfiguration of the deletion correction code used in System 100 when the number of error domains changes (increases or decreases). The flexible coded data management engine 106 is able to modify the deletion correction code without requiring a complete recalculation of the deletion correction codes based on the entire data collection stored in System 100. The flexible coded data management engine 106 supports the reconfiguration of deletion correction codes when System 100 is scaled up or down.

[0030] As used here, an “engine” can refer to a hardware processing circuit, which may include any or a combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuitry. Alternatively, an “engine” can refer to a combination of a hardware processing circuit and machine-readable instructions (software and / or firmware) that are executable on the hardware processing circuit.

[0031] As in Fig. Further described, System 100 includes a memory 108 that stores information 110 about a current erase correction configuration (using an existing erase correction code) used in System 100. The memory 108 can be implemented using a collection of storage devices (a single storage device or multiple storage devices). A storage device can include any or a combination of the following: a dynamic random-access memory device (DRAM), a static random-access memory device (SRAM), a flash memory device, and so on. In other examples, the information 110 about the current erase correction configuration can be stored in a different type of storage, such as disk-based storage, a solid-state drive, and so on.

[0032] The flexible coded data management machine 106 receives a request 112 to switch to a new deletion correction configuration (which uses a new deletion correction code that differs from the existing deletion correction code). The request 112 can be made by a user, a program, or a machine. For example, a user on a remote computer device over a network can make the request to the flexible coded data management machine 106 due to a change in the configuration of system 100, such as adding a new error domain or removing an existing error domain.

[0033] In some examples of the present disclosure, the flexible coded data management engine 106 can group several coded data blocks 114 into segments 116. A coded data block 114 is a block of coded data generated by applying a deletion correction coding to original data. The application of the deletion correction coding to the original data produces coded data that can be subdivided into coded data blocks 114 for storage across multiple error domains 102.

[0034] A “segment” can refer to a collection of coded data blocks (a single coded data block or multiple coded data blocks). In Fig. Each segment 116 comprises several coded data blocks 114. According to some implementations of the present disclosure, a segmentwise erase correction code (e.g., a segmentwise Reed-Solomon erase code) comprises n + m segments, where each segment q ≥ 1 contains coded data blocks and is contained in a corresponding error domain. In this arrangement, there are a total of nq + mq data blocks. In contrast, a traditional erase correction code (e.g., a traditional Reed-Solomon erase code) contains n + m coded data blocks. The segmentwise erase correction code can protect against the loss of up to m segments.

[0035] In one example, it is assumed that a Reed-Solomon erasure correction code is used to encode 12 data blocks, so that the encoded data comprises the following: 12 + 12 encoded data blocks b1 to b 24 (in this example, nq = 12 and mq = 12): [b1,b2,b3,…,b23,b24]

[0036] In one example, the data blocks encoded above are grouped into four segments, each containing the following data blocks (q = 6 data blocks): Segment 1=[b1,b2,b3,b4,b5,b6], Segment 2=[b7,b8,b9,b10,b11,b12], Segment 3=[b13,b14,b15,b16,b17,b18], and Segment 4=[b19,b20,b21,b22,b23,b24].

[0037] These four segments can be used to implement a 2+2 segment-wise erase correction code (n + m, where n = 2 and m = 2). In this erasure correction code, each segment contains multiple (q = 6 in the example) encoded data blocks, and the four segments are contained in four error domains each. The 2+2 segment-wise erase correction code can protect against the loss of up to 2 of the 4 segments, since any 2 segments contain 12 encoded data blocks, allowing the 12 original data blocks to be recovered.

[0038] When System 100 is extended by adding new error domain(s), the segment-wise erase correction code can be reorganized to change the erase correction configuration from the current erase correction configuration (which uses the existing erase correction code) to the new erase correction configuration (which uses the new erase correction code). The current erase correction configuration can use the 2+2 segment-wise erase correction code described above. If an additional error domain is added, the new erase correction configuration will use a 3+2 segment-wise erase correction code (n+m, where n = 3 and m = 2), in which the coded data blocks are regrouped into five segments as follows: Segment 1=[b1,b2,b3,b4], Segment 2=[b5,b6,b7,b8], Segment 3=[b9,b10,b11,b12], Segment 4=[b13,b14,b15,b16], and Segment 5=[b17,b18,b19,b20].

[0039] Note that these 5 segments are formed by simply rearranging the data blocks used in the 2+2 code described above and by discarding 4 coded data blocks, b 21 , b 22 , b 23 , b 24 After this reconfiguration, q = 4. If no more than 2 of these segments are lost, then the remaining 3 segments contain 12 data blocks and thus allow the recovery of the original data.

[0040] The five segments are contained in five error domains. If another error domain is added, resulting in six error domains, then a new deletion correction configuration uses a 4 + 2 segment-wise deletion correction code (n + m, where n = 4 and m = 2), in which the coded data blocks are regrouped into six segments (contained in the respective six error domains) as follows: Segment 1=[b1,b2,b3], Segment 2=[b4,b5,b6], Segment 3=[b7,b8,b9], Segment 4=[b10,b11,b12], Segment 5=[b13,b14,b15], and Segment 6=[b16,b17,b18].

[0041] Note that these 6 segments are formed by rearranging the data blocks in the 2+2 or 3+2 codes with two additional coded data blocks (b 19 and b 20 ) will be discarded, in addition to the 4 coded data blocks (b 21 , b 22 , b 23 , b 24 ), which were previously discarded. In this arrangement, q = 3. In other words, the above 4 + 2 segment-wise erase correction code does not use coded data blocks b. 19 to b 24 .

[0042] If the six error domains are increased to 14 error domains, then a new deletion correction configuration uses a 12 + 2 segment-wise deletion correction code (n + m, where n = 12 and m = 2), in which the coded data blocks are regrouped into 14 segments (contained in the respective 14 error domains) as follows: Segment 1=[b1], Segment 2=[b2], Segment 3=[b3], Segment 4=[b4], Segment 5=[b5], Segment 6=[b6], Segment 7=[b7], Segment 8=[b8], Segment 9=[b9], Segment 10=[b10], Segment 11=[b11], Segment 12=[b12], Segment 13=[b13], and Segment 14=[b14].

[0043] The above 12 + 2 segment-wise deletion correction code does not use coded data blocks. 19 to b 24 but can restore the original data if only 12 segments are retained.

[0044] In these examples, the reorganization of the segment-wise deletion correction code is achieved by simply copying existing coded data blocks and discarding some coded data blocks to form new segments of coded data blocks to support a new deletion correction code.

[0045] The example described above began with 24 coded data blocks (i.e., 12 + 12, where n = 12 and m = 12), coded such that 12 coded data blocks (e.g., the first n blocks) can be recovered from any 12 of the initial 24 coded data blocks. As can be seen above, each erase correction configuration contains at least 12 of the original 24 coded blocks, thus enabling the recovery of 12 original data blocks in each erase correction configuration.

[0046] A similar procedure can be used to convert an n + m segment delete code with p data blocks per segment into any n' + m' configuration with p' data blocks per segment, as long as n'p' ≥ np. Furthermore, if (n' + m')p' ≤ (n + m)p, then the conversion involves only reconfiguration and deletion of blocks. If (n' + m')p' > (n + m)p, then in addition to rearrangement, some blocks are also recalculated. For example, when converting from the 4+2 configuration with p = 3 to the 3+2 configuration with p' = 4, blocks b 19 and b 20 will be recalculated.

[0047] In the current deletion correction configuration, segments 116 are stored in an initial number of fault domains 102. The current deletion correction configuration enables fault recovery in response to lost segments in a specified number (m) of fault domains 102.

[0048] In response to request 112, the flexible coded data management engine 106 changes a deletion correction configuration from the current deletion correction configuration to the new deletion correction configuration by reorganizing segments 116 into further segments 202, as shown in Fig. depicted.

[0049] In the example of Fig. It is assumed that the initial number of error domains (before the change) is 4. It is assumed that one or more new error domains (104) are added (two new error domains (104) are present in the example of...). Fig. (as shown). With 4 fault domains 102, each comprising 4 segments 116, a 2+2 deletion correction code is supported. However, by adding two new fault domains 104, the system 100 is modified to support a second number of fault domains (6 in the example of Fig. ) contains, each comprising 6 segments 202, so that the flexible coded data management engine 106 can switch from the 2 + 2 deletion correction code (of the existing deletion correction configuration) to the 4 + 2 deletion correction code (of the new deletion correction configuration).

[0050] Note that each segment 202 contains a different number of coded data blocks 114 than each segment 116.

[0051] After the reconfiguration, the flexible coded data management engine 106 stores the segments 202 in the respective fault domains 102 and 104 (which form the second set of fault domains) to provide the new deletion correction configuration.

[0052] Although the examples discussed here relate to the application of erasure correction coding, the techniques or mechanisms of rearranging blocks of coded data into coded segments, as described in some implementations, can be applied to both general error coding and erasure coding. In such a more general case, it is usually unknown which segment might be corrupted, so only [m / 2] errors can be corrected, where [m / 2] is a function that returns the largest integer less than or equal to m / 2. For coded data produced by error correction coding containing n + m blocks, the identification and correction of the corrupted data can be performed by retaining n + [m / 2] blocks, where [m / 2] is a function that returns the smallest integer greater than or equal to m / 2. Reed-Solomon erase code

[0053] The Reed-Solomon erasure coding is described below.

[0054] The original data can be written as a matrix x with n rows, each containing k (k ≥ 1) values, where each value has w bits. For example, w = 8, so that each value in the original x, or the encoded data r, is an unsigned 8-bit integer (commonly known as a byte), and the original data x consists of n blocks, each containing k bytes.

[0055] In other examples, w can have a different non-zero value.

[0056] In this representation, each row of x can be considered a single data block. The coded data r is obtained by multiplying a coding matrix A with the original data x: Ax=r.

[0057] The encoding matrix A has n + m rows, so r is (n + m) × k bytes in size. Note that the encoded data has r × n + m rows (or equivalently, in some examples, n + m blocks).

[0058] In Reed-Solomon erasure coding, the arithmetic can be performed in the modular Galois field GF(2). w ) are performed, where the addition is an exclusive OR operation and the multiplication is performed modulo a primitive polynomial.

[0059] In some examples, A represents a systematic code where the first n values ​​in r are identical to the values ​​in x. A systematic code is a code that, when applied to input data, produces an output code that contains a part consisting of the input data and another part containing the redundant information.

[0060] If A represents a systematic code, then A can be expressed by the following structure: A=[InFm,n], where I n an identity matrix with the following rows has n rows, and F m,n a specific coding matrix. The identity matrix I n Multiplying by the original data x yields the first n lines of the coded data r, which are equal to the n lines of the original data x.

[0061] The website F m,n (with m rows and n columns) multiplied by the original data x yields m rows of parity data p.

[0062] The multiple of [InFm,n] (the systematic code A) with the original data x yields the following coded data r: r=[xp]=[xFm,nx].

[0063] Effectively, the encrypted data r can be written as a concatenation of the original data x and the parity data p generated by F. m,n x.

[0064] If the parity data p is small relative to x (the number of rows, m, of the parity data p is much smaller than the number of rows, n, of the original data x), the calculation p = F m,n x can be performed faster than would be the case with a non-systematic code A.

[0065] However, in other examples of the present revelation, A may be an unsystematic code.

[0066] In some examples, F m,n a partial Vandermonde matrix, as follows: Fm,n=[111…1123…n⋮⋱⋮1m2m3m…nm].

[0067] Vandermonde matrices in the modular Galois field GF(2 w ) have the property that if n + m < 2 wthen rank(A) = n. Indeed, the rank of any matrix A' that can be obtained by selecting any n rows from A also has the rank n. The fact that every n × n submatrix of A is complete is useful in deletion correction, because if any n rows are selected from the n + m rows of the encoded data r, the following results: A'x=r'. where the rows of A' (which is a submatrix of A) are combined with the rows of r' (which is a subset of r).

[0068] A matrix is ​​complete if all rows and columns of the matrix are linearly independent.

[0069] Since A' is fully-fledged, A' is invertible, and the system can be solved for x if the value of r' is known and the n rows have survived (i.e., the n encoded data blocks of the error domains have not been lost), the practical consequence is that if one of the n of the n + m rows of encoded data r survives, the entire value of the original data x can be recovered. Furthermore, since m can be small (e.g., much smaller than n), A' can be nearly equal to the identity matrix, allowing a complete LU decomposition to derive x based on A' and r' to be performed efficiently. In particular, because A represents a systematic code, the LU decomposition of A' can be performed by decomposing all the rows of F. n,m They survive in A'. The rest of A' remains unchanged. Flexible Reed-Solomon erase code

[0070] The following describes some examples of the flexible Reed-Solomon erase coding according to some embodiments of the present disclosure.

[0071] The flexible Reed-Solomon erase coding provided by the flexible coded data management engine 106 according to some examples in the present disclosure allows the erasure code to be changed with far fewer computations than are required to recalculate the coded data from the entire collection of original data.

[0072] In some examples, for an n + m Reed-Solomon erase code, m can be set to 1, 2, or 3, and n ≤ 12. Larger values ​​of n (greater than 12) can lead to negligible improvements in coding efficiency. The above restrictions allow the construction of a family of related erase codes based on Reed-Solomon-encoded data, which can begin with n = 2, where n is increased to a value of, for example, 3, 4, 6, or 12 by simply rearranging encoded data blocks (and discarding some encoded data blocks).

[0073] Increasing the value of n can be done in response to an increase in the number of fault domains, for example by adding server computers or storage devices to System 100.

[0074] Although specific examples for n and m are given, it should be noted that in other examples different values ​​of n and / or m may be used by the flexible coded data management machine 106 according to some implementations of the present disclosure.

[0075] In a concrete example, the management engine 106 for flexible coded data can define a basic encoding structure of a 12 + 12 Reed-Solomon erase code, expressed line by line as follows: [I1⋮I12F12,12,1⋮F12,12,12][x1⋮x12]=[x1⋮x12r1⋮r12] where x1 to x 12 The 12 rows (or blocks) of the original data are x, I1 to I 12 the 12 rows of the identity matrix I 12 (an identity matrix with 12 rows), F 12,12,1 to F 12,12,12 are 12 submatrices of F 12,12 Matrix (the F 12,12 The matrix has 12 rows and 12 columns), and r1 to r 12are the 12 rows of parity data p generated by F m,n x. Here I j for the j-row of the identity matrix I n and F 12,12,j stands for the j-row of F 12,12 that, namely [1 j , 2 j , ..., n j ].

[0076] The above representation of the 12 + 12 Reed-Solomon erase code can be rearranged segment by segment as follows: [[I1⋮I6][I7⋮I12][F12,12,1⋮F12,12,6][F12,12,7⋮F12,12,1 2]][[x1⋮x6][x7⋮x12]]=[[x1⋮x6][x7⋮x12][r1⋮r6][r7⋮r12]] where [x1⋮x6],[x7⋮x12],[r1⋮r6], and [r7⋮r12] There are four segments, each with 6 coded data blocks. The 6 "coded data blocks" of each segment [x1⋮x6] and [x7⋮x12] are actually original data blocks, while the 6 "encrypted data blocks" of the segments [r1⋮r6] and [r7⋮r12] Each of these are parity blocks (blocks of p).

[0077] Each of the four segments [x1⋮x6],[x7⋮x12],[r1⋮r6], and [r7⋮r12] can be an example of a in Fig. The segment shown is 116.

[0078] The above form of the 12 + 12 Reed-Solomon erase code can be written in abbreviated form as follows: [I1…6I7…12F1…6F7…12][X1…6X7…12]=[X1…6X7…12R1…6R7…12], where X1…6=[x1⋮x6],,X7…12=[x7⋮x12],R1…6=[r1⋮r6], and R7…12=[r7⋮r12].

[0079] The above form, [X1…6X7…12R1…6R7…12] With four segments (each in 4 error domains), it can be considered a 2+2 erase code. The four segments are Segment X. 1...6 , Segment X 7...12 , Segment R 1...6 and segment R 7...12 .

[0080] If a new fault domain is added to bring the total to 5 fault domains, then the above deletion code can be used. [X1…6X7…12R1…6R7…12] as follows: restructured into 5 corresponding segments X 1...4 , X 5...8 , X9...12 , R 1...4 , and R 5...8 : [I1…4I1…8I9…12F1…4F5…8][X1…4X5…8X9…12]=[X1…4X1…8X9…12R1…4R5…8].

[0081] The 5 segments can serve as an example of the in Fig. The segments shown are 202.

[0082] Note that in the 5 segments, rows r9 on r 12 The parity data p were discarded, and only 8 rows of the coded data (x1 to x) were retained. 12 and r1 to r8). Note also that only 8 rows of F are used.

[0083] Each of the 5 segments X 1...4 , X 5...8 , X 9...12 , R 1...4 , and R 5...8 Contains 4 coded data blocks. These 5 segments result in a 3 + 2 deletion code.

[0084] Note that the 2+2 delete code can be converted to the 3+2 delete code by simply rearranging the rows of the original data, x1 to x 12 and the discarding of 4 lines from p.

[0085] A similar procedure can be used to create a 4+2 delete code from the 3+2 delete code, as described below: [I1…3I4…6I7…9I10…12F1…3F4…6][X1…3X4…6X7…9X10…12]=[X1…3X4…6X7…9X10…12R1…3R4…6], in which lines r7 to r 12 The parity data p were discarded, and only 6 lines of the encrypted data (x1 to x) were retained. 12 and r1 to r8). Note also that only 6 rows of F are used.

[0086] The output of this 4+2 erase code (including 6 segments) can be derived from the output of the 3+2 erase code by rearranging the rows of the original data, x1 to x 12 and discarding 6 rows of parity data p. When converting from the 3 + 2 delete code to the 4 + 2 delete code, the 5 segments X are 1...4 , X 5...8 , X 9...12 , R 1...4 , and R 5...8 which correspond to the 3+2 deletion code, as segments 116 of Fig. be considered, and the 6 segments X 1...3 , X 4...6 , X 7...9 , X 10...12 , R 1...3 , and R 4...6 Those corresponding to the 4+2 deletion code are designated as segments 202 of Fig. considered.

[0087] A similar procedure can be used to create a 6+2 deletion code from the 4+2 deletion code and a 12+2 deletion code from the 6+2 deletion code.

[0088] In accordance with some implementations of the present disclosure, the transformations from n = 2 to n = 3 and finally to n = 12 will only involve copying and deleting and no computations of encoded data.

[0089] Conversely, reducing the number of error domains (e.g., by removing one or more error domains) may require a small amount of computational effort. For example, a 4+2 erase code can be converted into a 3+2 erase code, where the 6 segments X 1...3 , X 4...6 , X 7...9 , X 10...12 , R 1...3 , and R 4...6 which are the 4 + 2 segments 116 of Fig. be considered, and the 5 segments X 1...4 , X 5...8 , X 9...12 , R 1...4 , and R 5...8 Those corresponding to the 3 + 2 deletion code are considered segments 202 of Fig. considered.

[0090] Note that the 6 segments X 1...3 , X 4...6 , X 7...9 , X 10...12 , R 1...3 , and R 4...6According to the 4+2 delete code, the missing lines are r7 and r8 of the parity data p, since these lines were discarded during the transition to the 4+2 delete code. Consequently, lines r7 and r8 of the parity data p, which are part of the 5 segments X, are 1...4 , X 5...8 , X 9...12 , R 1...4 , and R 5...8 the 3 + 2 deletion code.

[0091] Note that only two rows of the parity data p are calculated in this example, instead of recalculating all rows of the parity data p. Further implementation examples

[0092] Fig. A block diagram of a non-transitory machine-readable or computer-readable storage medium 300, which stores machine-readable instructions that, when executed, cause a system to perform various tasks. The system can comprise one computer or multiple computers.

[0093] The machine-readable instructions include first segment grouping instructions 302 to group a large number of blocks of coded data into first segments (e.g., segments 116 in Fig. ), where each first segment contains several blocks of the plurality of blocks and the encoded data is based on the application of a deletion correction encoding to input data.

[0094] The machine-readable instructions include first segment storage instructions 304 to store the first segments in respective error domains of a first set of error domains in a first erase correction configuration, wherein the first erase correction configuration enables error recovery in response to lost coded data in a certain number (m) of the plurality of error domains.

[0095] The machine-readable instructions include instruction 306 for modifying the deletion correction configuration to change a deletion correction configuration from the first deletion correction configuration to a second deletion correction configuration by reorganizing the first segments into second segments (e.g., 202 in Fig. ). Every second segment contains a different number of blocks of coded data than the first segment.

[0096] In some examples, the reorganization of the first segments into the second segments to switch to the second deletion correction configuration is performed without recalculating the coded data based on the input data.

[0097] In some examples, the reorganization discards a number of the multitude of blocks of coded data to produce a remainder of the multitude of blocks of coded data, with the machine-readable instructions grouping the remainder of the multitude of blocks of coded data into the second segments.

[0098] The machine-readable instructions contain memory instructions for second segments 308 to store the second segments in corresponding fault domains of a second set of fault domains in the second deletion correction configuration, where the second set is different from the first set.

[0099] In some examples, the first segments according to the first erasure correction configuration comprise n1 + m first segments, where n1 represents the number of input data segments that can be recovered if up to m segments of lost encoded data are present. The second segments according to the second erasure correction configuration comprise n2 + m first segments, where n2 > n1 and n2 represents the number of input data segments that can be recovered if up to m segments of lost encoded data are present.

[0100] In some examples, the machine-readable instructions change the deletion correction configuration from the second deletion correction configuration to a third deletion correction configuration by reorganizing the second segments into third segments, with each third segment of the third segments containing a different number of blocks of encoded data than a second segment of the second segments.

[0101] In some examples, the third deletion correction configuration includes a third number of error domains, where the third number differs from the second number.

[0102] In some examples, the machine-readable instructions change the deletion correction configuration from the second deletion correction configuration to a third deletion correction configuration, which is associated with a third set of error domains that is smaller than the second set of error domains, where changing the deletion correction configuration from the second deletion correction configuration to the third deletion correction configuration involves copying the blocks of coded data in the second segments to third segments and recalculating blocks of coded data to add to the third segments.

[0103] In some examples, the recalculation of the blocks of coded data to be added to the third segments involves recalculating less than the multitude of blocks.

[0104] In some examples, every third segment of the third segments has a larger number of blocks of coded data than a second segment of the second segments.

[0105] In some examples, the first set of fault domains includes a first set of server computers or a first set of storage devices, and the second set of fault domains includes a second set of server computers or a second set of storage devices.

[0106] In some examples, the multitude of blocks of coded data comprises a first set of blocks of input data and a second set of blocks of parity data, which are calculated based on the application of a coding matrix to the input data.

[0107] Fig. This is a block diagram of a System 400 according to some examples.

[0108] The System 400 includes one or more hardware processors (402). A hardware processor can be a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuitry.

[0109] The System 400 includes a storage medium 404 on which machine-readable instructions are stored that can be executed on the hardware processor 402 to perform various tasks. Machine-readable instructions executable on a hardware processor can refer to instructions that can be executed on a single hardware processor or on multiple hardware processors.

[0110] The machine-readable instructions in the storage medium 404 contain first segment grouping instructions 406 to group a plurality of blocks of coded data into a first number of segments, wherein each segment of the first number of segments contains several blocks of the plurality of blocks and the coded data is based on the application of an error correction coding (e.g., a deletion correction coding or some other error correction coding) to input data.

[0111] The machine-readable instructions in storage medium 404 include first segment storage instructions 408 to store the segments of the first number of segments in corresponding fault domains of a first set of fault domains in a first fault correction configuration, where the first number of segments supports data recovery in response to a certain number of lost segments of the first number of segments.

[0112] The machine-readable instructions in storage medium 404 contain instructions 410 for changing the error correction configuration to change an error correction configuration from the first error correction configuration to a second error correction configuration by reorganizing the first number of segments into a second number of segments, each segment of the second number of segments containing a different set of blocks of coded data than a segment of the first number of segments.

[0113] The machine-readable instructions in the storage medium 404 include second segment storage instructions 412 to store the segments of the second number of segments in respective fault domains of a second set of fault domains in the second fault correction configuration, wherein the second set is different from the first set and a set of segments in the second number of segments is different from a set of segments in the first number of segments, and wherein the second number of segments supports data recovery in response to the specified number of lost segments of the second number of segments.

[0114] Fig. is a flowchart of a Process 500 that can be performed by a system with a hardware processor.

[0115] Process 500 involves grouping (at 502) a plurality of blocks of erase-encoded data into first segments, with each first segment of the first segments containing multiple blocks of the plurality of blocks, and the erase-encoded data being based on the application of Reed-Solomon erase coding to input data.

[0116] Process 500 involves storing (at 504) the first segments in corresponding error domains of an initial set of error domains in an initial erase correction configuration. The initial erase correction configuration enables error recovery in response to lost erase-encoded data in a specified number (m) of error domains.

[0117] Process 500 involves changing (at 506) a deletion correction configuration from the first deletion correction configuration to a second deletion correction configuration by reorganizing the first segments into second segments, where each second segment of the second segments contains a different number of blocks of deletion-encoded data than a first segment of the first segments.

[0118] In some examples, the change to the delete correction configuration occurs in response to a change in the number of fault domains due to the addition or removal of storage hardware (e.g., server computer, storage device, etc.).

[0119] Process 500 includes storing (at 508) the second segments in corresponding error domains of a second set of error domains in the second erase correction configuration, wherein the second set is different from the first set, and wherein the second erase correction configuration enables error recovery in response to lost erase-encoded data in the specified number (m) of error domains.

[0120] A storage medium (e.g., 300 in Fig. in Fig.The storage device may include any or a combination of the following: a semiconductor storage device such as dynamic or static random-access memory (DRAM or SRAM), erasable and programmable read-only memory (EPROM), electrically erasable and programmable read-only memory (EEPROM), and flash memory; a magnetic disk such as a hard disk, floppy disk, and removable disk; another magnetic medium, including tape; an optical medium such as a compact disc (CD) or digital video disc (DVD); or some other type of storage device. It should be noted that the instructions described above may be provided on a single computer- or machine-readable storage medium, or alternatively, on multiple computer- or machine-readable storage media distributed throughout a large system, possibly with multiple nodes.Such a computer-readable or machine-readable storage medium or media are considered part of an article (or a manufactured item). An article or manufactured item may refer to any single manufactured component or to multiple components. The storage medium or media may be located either in the machine on which the machine-readable instructions are executed or at a remote location from which machine-readable instructions can be downloaded for execution over a network.

[0121] The foregoing description includes numerous details to provide an understanding of the subject matter disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and deviations from the details described above. It is intended that the accompanying claims cover such modifications and variations.

Claims

[1] A non-transitory, machine-readable storage medium (300) containing instructions which, when executed, cause a system to: to group a plurality of blocks of coded data (114) into first segments (302), wherein each first segment of the first segments comprises several blocks of the plurality of blocks and the coded data is based on an application of a deletion correction coding on input data; to store the first segments in respective error domains of a first set of error domains (102) in a first erase correction configuration (304), wherein the first erase correction configuration enables error recovery in response to lost encoded data in a certain number of error domains; to change a deletion correction configuration from the first deletion correction configuration to a second deletion correction configuration by reorganizing the first segments into second segments (306), wherein every second segment of the second segments comprises a different set of blocks of coded data (114) than a first segment of the first segments; to store the second segments in respective error domains of a second set of error domains (104) in the second deletion correction configuration (308), wherein the second set is different from the first set; and to change the deletion correction configuration from the second deletion correction configuration to a third deletion correction configuration that is associated with a third set of fault domains that is smaller than the second set of fault domains, wherein changing the deletion correction configuration from the second deletion correction configuration to the third deletion correction configuration includes the following: Copying the blocks of coded data (114) in the second segments into the third segments, and Recalculating blocks of coded data to add to the third segments. [2] The non-transitory, machine-readable storage medium (300) according to claim 1, wherein the first segments according to the first erasure correction configuration comprise n1 + m first segments, where m represents the specified number, and n1 represents a number of segments of the input data that are recoverable when up to m segments of lost encoded data are present, and wherein the second segments according to the second erasure correction configuration comprise n2 + m second segments, where n2 > 1, and n2 represents a number of segments of the input data that are recoverable when up to m segments of lost encoded data are present. [3] The non-transitory, machine-readable storage medium (300) according to claim 2, wherein the reorganization of the first segments into the second segments to switch to the second erase correction configuration is performed without recalculating encoded data based on the input data. [4] The non-transitory, machine-readable storage medium (300) according to claim 3, wherein the reorganization discards a number of the plurality of blocks of encoded data in order to generate a remainder of the plurality of blocks of encoded data, and wherein the instructions during execution cause the system to: to group the rest of the multitude of blocks of coded data into the second segments. [5] The non-transitory, machine-readable storage medium (300) according to claim 1, wherein the erase correction encoding comprises a Reed-Solomon erase encoding. [6] The non-transitory, machine-readable storage medium (300) according to claim 1, wherein the plurality of blocks of coded data grouped in the first segments comprises 12 blocks of coded data. [7] The non-transitory, machine-readable storage medium (300) according to claim 1, wherein each third segment of the third segments has a larger number of blocks of coded data than a second segment of the second segments. [8] The non-transitory, machine-readable storage medium (300) according to claim 1, wherein the recalculation of blocks of coded data for adding to the third segments comprises recalculation of less than the plurality of blocks. [9] The non-transitory, machine-readable storage medium (300) according to claim 1, wherein the first set of fault domains comprises a first set of server computers or a first set of storage devices and the second set of fault domains comprises a second set of server computers or a second set of storage devices. [10] The non-transitory, machine-readable storage medium (300) according to claim 1, wherein the plurality of blocks of coded data comprises a first number of blocks of input data and a second number of blocks of parity data calculated based on applying a coding matrix to the input data. [11] A system comprising (400): a processor (402); and a non-transitory storage medium (404) that stores instructions that can be executed on the processor to: to group a plurality of blocks of coded data (114) into a first number of segments (406), wherein each segment of the first number of segments comprises several blocks of the plurality of blocks and the coded data is based on an application of an error correction coding to input data; to store the segments of the first set of segments in respective error domains of a first set of error domains (102) in a first error correction configuration (408), wherein the first set of segments supports data recovery in response to up to a certain number of lost segments of the first set of segments, to change an error correction configuration from the first error correction configuration to a second error correction configuration by reorganizing the first number of segments into a second number of segments (410), wherein each segment of the second number of segments comprises a different set of blocks of coded data than a segment of the first number of segments; and to store the segments of the second set of segments in respective error domains of a second set of error domains (104) in the second error correction configuration (412), wherein the second set is less than the first set and a set of segments in the second set of segments is less than a set of segments in the first set of segments, wherein the second set of segments supports data recovery in response to up to the specified number of lost segments of the second set of segments, and wherein reorganizing the first set of segments into the second set of segments to transition into the second error correction configuration includes: Copying the blocks of coded data (114) in the first number of segments into the second segment, and Recalculating blocks of coded data to add to the second segments. [12] A method (500) for a system comprising a hardware processor, comprising: Grouping (502) a plurality of blocks of erase-encoded data into first segments, wherein each first segment of the first segments comprises several blocks of the plurality of blocks and the erase-encoded data is based on an application of a Reed-Solomon erase encoding to input data; Storing (504) the first segments in respective error domains of a first set of error domains (102) in a first erase correction configuration, wherein the first erase correction configuration enables error recovery in response to lost erase-coded data in a certain number of error domains; Modifying (506) a deletion correction configuration from the first deletion correction configuration to a second deletion correction configuration by reorganizing the first segments into second segments, wherein every second segment of the second segments comprises a different set of blocks of deletion-encoded data than a first segment of the first segments; and Storing (508) the second segments in respective error domains of a second set of error domains (104) in the second erase correction configuration, wherein the second set is less than the first set and wherein the second erase correction configuration enables error recovery in response to lost erase-coded data in the specified number of error domains, and wherein reorganizing the first segments into the second segments to transition into the second erase correction configuration includes: Copying the blocks of erase-encoded data in the first segments to the second segment, and Recalculating blocks of erasure-encoded data to add to the second segments. [13] The method (500) according to claim 12, wherein the change of the deletion correction configuration is performed in response to a change in the set of fault domains due to the addition or removal of memory hardware.

Images