Data recovery system

Abstract

To provide a data recovery system which can create a large number of redundant data units and thus can recover from damage to a storage device with high probability by restoring a large number of original data.SOLUTION: A data recovery system provided herein is configured to create multiple original data units and multiple redundant data units from original data, extract minimum required redundant data units that have escaped damage, and restore the original data units and redundant data units efficiently from the extracted redundant data units through computation using a matrix of a 32-bit Galois field.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a data recovery system configured to create and store data that should exist (hereinafter referred to as "original data") and redundant data intended to restore the original data. [Background technology]

[0002] Recording and playback devices that use physical drives such as HDDs (Hard Disk Drives) are known to be susceptible to damage to data written on the HDD due to HDD failures caused by motor or disk deterioration, laptop computer failures caused by battery failure or unintentional data deletion or modification, storage device failures caused by power outages or natural disasters, and even wireless communication failures.

[0003] Under these circumstances, a technology known as RAID (Redundant Arrays of Independent (Inexpensive) Disks) is used to distribute and store various types of data across multiple disks, improving the reliability, recoverability, and speed of recording and playback devices that use physical drives.

[0004] In addition to data disks, RAID also includes parity disks for redundancy, and when a failure occurs in a data disk, the data lost due to the failure is restored using redundant data.

[0005] When a failure occurs in a data disk, the data lost due to the failure is restored by performing an exclusive OR on a unit basis (see Patent Document 1 below). To restore data, for example, the original data D is divided into multiple (n) partial data as shown below, and k linearly independent vectors α are extracted from a finite field called a Galois field. 11 ,α 21 ,α 31 ,……α n1 and generate parity data using equation (1). In addition, the original data D={D1, D2, D3, ..., D n}, Transmission data D′={C1, C2, C3, …, C m}, Parity data Ck=α 11 D1+α 21 D2+...+α n1 D n is.

[0006] When restoring the original data D, if n partial data are obtained, the n × n matrix A = (α ij ), 1≦i≦n, 1≦j≦n, and the original data can be obtained by multiplying the obtained partial data by the inverse matrix of A. In other words, the original data is restored using formula (2).

[0007]

number

[0008]

number

[0009] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-172617 Summary of the Invention [Problem to be solved by the invention]

[0010] However, for example, in conventional RAID6, the number of parity disks is fixed at two, so if three or more original data disks are damaged, the original data disks cannot be restored even if no parity disks are damaged.

[0011] The present invention has been made in view of the above circumstances, and has as its object to provide a data recovery system that can recover data even if a large number of original data disks are damaged. [Means for solving the problem]

[0012] In order to solve the above problems, the data recovery system according to the present invention includes a redundant unit creating means for creating R (R is a natural number greater than or equal to 2) redundant data units Q[0] to Q[R-1] for recovering M (M is a natural number greater than or equal to 2) original data units D[0] to D[M-1], and an original unit recovering means for recovering the damaged L original data units from R K (a natural number) undamaged redundant data units when L (a natural number) of the original data units are damaged, wherein the redundant unit creating means: R redundant data units Q[0] to Q[R-1] are created from M original data units D[0] to D[M-1] through the calculation process of formula (3), and the original unit restoration means extracts L uncorrupted redundant data units to be used to restore the original data units when L≦RK, and restores the original data units from the extracted L redundant data units Q[a[0]], ..., Q[a[L-1]] and ML uncorrupted original data units D[y[0]], ..., D[y[ML-1]] through the calculation process of formula (4).

[0013]

number

[0014]

number

[0015] A configuration can be adopted which includes a redundant unit restoration means which, after restoring the original data units using the original unit restoration means, restores corrupted redundant data units Q[b[0]], ..., Q[b[K-1]] from all original data units [0], ..., D[M-1] including the restored original data units through the calculation process of equation (5).

[0016]

number

[0017] In the above calculation process, f is calculated for the 32nd degree polynomial of the Z / 2Z coefficient (Equation (6)). 32 The present invention includes a multiplication unit that multiplies an arbitrary element a (Equation (8)) of a 32-bit Galois field generated by α, which is expressed by Equation (7), with element b (Equation (9)) as the solution of (x)=0, to derive a product ab=c, and the multiplication unit includes a calculation unit that obtains a 62nd degree polynomial C(x) (Equation (12)) which is the product of A(x) (Equation (10)) and B(x) (Equation (11)) in which α of the elements a and b is substituted for variable x, and a calculation unit that calculates a 62nd degree polynomial C(x) (Equation (12)) in which α of the elements a and b is substituted for variable x, and a calculation unit that calculates a 62nd degree polynomial C(x) (Equation (13)) in which α of the elements a and b is substituted for variable x, and a calculation unit that calculates a 62nd degree polynomial C(x) (Equation (14)) in which α of the elements a and b is substituted for variable x, and a calculation unit that calculates a 62nd degree polynomial C(x) (Equation (15)) in which α of the elements a and b is substituted for variable x, and a calculation unit that calculates a 62nd degree polynomial C(x) (Equation (16)) in which α of the elements a and b is substituted for variable x, and a calculation unit that calculates a 62nd degree polynomial C(x) (Equation (17)) in which α of the elements a and b is substituted for variable x, and a calculation unit that calculates a 62nd degree polynomial C(x) (Equation (18)) in which α of the elements a and b is substituted for variable x, and a calculation unit that calculates a 62nd degree polynomial C(x) (Equation k When the exponent of is 32≦k, the polynomial is converted into a 31st degree polynomial (Equation (14)) using coefficients e(k,j) (0≦k≦62, 0≦j≦31) by using Equation (13), and the α k a data table for storing values ​​of a matrix whose components are the coefficients (e(k,31) e(k,30) ... e(k,1) e(k,0)) of C(x), and a data table for storing values ​​of a matrix whose components are the coefficients (c 62 … c 32 c 31 . . c1c0) and the matrix held in the data table in the form of a product of the components thereof.

[0018]

number

[0019]

number

[0020]

number

[0021]

number

[0022]

number

[0023]

number

[0024]

number

[0025]

number

[0026]

number

[0027] In the above calculation process, f is calculated for the 32nd degree polynomial of the Z / 2Z coefficient (Equation (6)). 32 Let the solution of (x) = 0 be α, and for any element a (Equation (8)) of the 32-bit Galois field generated by α that satisfies Equation (7), n (0≦n≦2 32 -1), and the exponentiation means includes a decomposition means for decomposing the number of times n to be exponentiated of the exponentiation object a into the formula (35), and a coefficient n in binary notation of the number of times n to be exponentiated based on the result obtained by the decomposition means. j n out of (0≦j≦31) jFor j where is "1", in each term of Equation (15), α in Equation (14) 31 From α 0 The coefficient of k=31*2 j , k=30*2 j , …,k=1*2 j ,k=0*2 j The fitted e(31*2 j ,31)…e(31*2 j ,0)…e(0*2 j ,31)…e(0*2 j ,0) is the matrix P j A data table that holds the value of (0≦j≦31) and the coefficient n of the binary notation of the exponentiation number n j n out of (0≦j≦31) j All a^2 for which is "1" j The coefficients of each term of the exponent a (Equation (16)) are 31 ...a1a0) and the matrix P whose elements are the values ​​stored in the data table j The product of (a 31 …a1a0)P j and a shaping means for outputting all a^2 output from the shaping means. j The configuration may include a multiplication means for multiplying and outputting the result. Furthermore, the arithmetic processing may be configured to include both the multiplication means and the exponentiation means.

[0028]

number

[0029]

number

[0030]

number

[0031]

number

[0032]

number

[0033]

number

[0034]

number

[0035] According to the data recovery system of the present invention, the number of redundant data units can be increased as needed, thereby achieving a high recovery probability. Furthermore, the number of redundant data units can be appropriately adjusted depending on the susceptibility of the original data units to corruption. As a result, recovery from corruption occurring in the data in the storage device can be achieved with a high probability. [Brief explanation of the drawings]

[0036] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of a data restoration system according to the present invention. [Figure 2] 1 is a flowchart showing the flow of processing performed in the data recovery system according to the present invention. [Figure 3] 1 is a block diagram showing an example of the functional configuration of a data restoration system according to the present invention; [Figure 4] FIG. 10 is an explanatory diagram showing an example of a redundant unit creation process used in the data restoration system according to the present invention. [Figure 5] FIG. 10 is an explanatory diagram showing an example of unit extraction processing (Case A) in the data restoration system according to the present invention. [Figure 6] FIG. 10 is an explanatory diagram showing an example (Case A) of an original unit restoration process in the data restoration system according to the present invention. [Figure 7]FIG. 1 is an explanatory diagram showing an example (Case A) of a redundant unit restoration process in a data restoration system according to the present invention. [Figure 8] FIG. 10 is an explanatory diagram showing an example of unit extraction processing (Case B) in the data restoration system according to the present invention. [Figure 9] FIG. 10 is an explanatory diagram showing an example (Case B) of an original unit restoration process in the data restoration system according to the present invention. [Figure 10] FIG. 10 is an explanatory diagram showing an example of unit extraction processing (Case C) in the data restoration system according to the present invention. [Figure 11] FIG. 10 is an explanatory diagram showing an example (Case C) of an original unit restoration process in the data restoration system according to the present invention. [Figure 12] FIG. 10 is an explanatory diagram showing an example (Case C) of a redundant unit restoration process in the data restoration system according to the present invention. [Figure 13] FIG. 10 is an explanatory diagram showing an example of unit extraction processing (Case D) in the data recovery system according to the present invention. [Figure 14] FIG. 10 is an explanatory diagram showing an example (Case D) of an original unit restoration process in the data restoration system according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0037] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a data recovery system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. The data restoration method and system described here use a 32-bit Galois field generated by a 32nd degree polynomial (Equation (6)) with Z / 2Z coefficients as follows: f 32 If we let α be the solution to (x)=0, we get equation (7).

[0038]

number

[0039]

number

[0040] The 32-bit Galois field is expressed by the formula (17) and #(GF(2 32 ))=2 32 These coefficients are expressed as (s 31 ,s 30 , …,s1,s0). In computer processing, (Z / 2Z) 32 Calculate as follows.

[0041]

number

[0042]

number

[0043] [The set of powers of α and GF(2 32 ) Relationship When i≦31, the power of α is α i is clearly GF(2 32 ) is included. When 32≦i, α can be obtained by repeating the following conversion: i is always GF(2 32 ) is satisfied. That is, when the exponent of α is 32 or more, α is calculated as in equation (18) using equation (13). 32 By repeating the substitution, α i ∈GF(2 32 ) can be said.

[0044]

number

[0045]

number

[0046] [Properties of α] Also, 1≦∀i≦2 32 In the range of -2, α i ≠1, α i ≠0 and equation (19) holds. However, this is the generator polynomial f 32 This depends on the choice of (x), and if another polynomial is chosen, this condition will generally not hold. That is, 1≦∀i≦2 32 -2 is alpha i ≠1, so 1≦i <j≦2 32 -2⇒α i ≠α j (※).

[0047]

number

[0048] [Set of powers of alpha] From the above conditions, #({α i |0≦i≦2 32 -2}∪{0})=2 32 [The set of powers of α and GF(2 32 ) relationship] As mentioned above for the case of 32≦i, and i |0≦i≦2 32 -2}∪{0}⊂GF(2 32 ) and the number of elements in the set mentioned above, {α i |0≦i≦2 32 -2}∪{0}=GF(2 32 ) can be said. That is, GF(2 32 ) can be thought of as the union of the powers of α and zero.

[0049] <Addition> [Addition of polynomials of α] The addition of polynomials of α is as shown in the following equation (20).

[0050]

number

[0051] [(Z / 2Z) 32 Addition in (Z / 2Z) 32 The addition is as shown in equation (21). [Relationship between two additions] Addition of polynomials of α and (Z / 2Z) 32 The relationship with the addition in is as shown in equation (22). The coefficients are divided as shown in equation (23). In computer processing, (Z / 2Z) 32 Calculate as follows.

[0052]

number

[0053]

number

[0054]

number

[0055] <multiplication> 32-bit Galois field GF(2 32 ) to obtain the product c from any element a and element b of [Multiply by] The element a, the element b, and the product c are as shown in Equation (24). [Polynomial in variable x] In this case, the 32-bit Galois field GF(2 32 ) generator α is set as variable x, and (Z / 2Z) 32 Coefficients a0~a 31 Let A(x) and (Z / 2Z) be polynomials of x. 32 The coefficients b0 to b 31 Let us create a polynomial B(x) of x, and the product of the two, A(x)B(x), is (Z / 2Z). 32 The coefficients c0 to c 62 Derive a polynomial C(x) of x.

[0056]

number

[0057] [Multiplication of polynomials of variable x] Here, we use multiplication of polynomials with the following variable x. First, regardless of α, the coefficient bit c 62 , …, c0, they are first expanded into polynomials of the variable x as shown in equations (10) to (12), and then calculated using the equation for the variable x (arithmetic processing).

[0058]

number

[0059]

number

[0060]

number

[0061] [Assign α to variable x] Substituting α for the variable x in the polynomial gives us equation (27).

[0062]

number

[0063] [Multiplication (converting the power of alpha to a 31st degree expression)] α k When the exponent of is 32≦k, the 31st degree polynomial using the coefficient e(k,j) that reduces the polynomial to a 31st degree polynomial using Equation (13) is determined to be Equation (14). That is, e(k,j):=α k When expressed as a 31st power of α, jAssuming that the coefficients are (0≦k≦62, 0≦j≦31), the following equation (28) is obtained (degree conversion processing).

[0064]

number

[0065]

number

[0066]

number

[0067] [Matrix multiplication] Let the matrix be as shown in the following formula (29), and then use GF(2 32 )⇔(Z / 2Z) 32 By identifying the above, the multiplication result of the 32-bit Galois field is derived. In the data recovery system, the multiplication process using this formula (28) is implemented in a computer to form a multiplication means. The multiplication means includes a calculation means for performing the calculation process, an order conversion means for performing the order conversion process, a data table for storing the values ​​of the matrix in the right part of the bottom row of formula (29), and a data table for converting C(α) into the coefficient (c 62 … c 32 c 31 …c1c0) and the α stored in the data table k and a shaping means for outputting the product of the components of a matrix having the coefficients (e(k,31) e(k,30) ... e(k,1) e(k,0)) of the above equation. The matrix is ​​a pre-calculated α k By creating and preparing a data table of 32 bits x 64 (including one dummy) = 256 bytes consisting of coefficients (0≦k≦62) (coefficients of the coefficients of C(x)), it is possible to perform multiplication reliably using less memory than previous multiplication processes.

[0068]

number

[0069] <Power> [Squaring mapping] A squaring map is a "linear map" that preserves the sum and preserves the coefficients Z / 2Z. It is as shown in equation (30), and 1 2 =1,0 2 Because =0.

[0070]

number

[0071] [2 j Multiplication map] From the above, 2 j The multiplication is also homomorphic (0≦∀j). However, in GF(2 32 ) is as shown in equation (31). In the end, we can restrict 0≦∀j≦31.

[0072]

number

[0073] [2 j matrix corresponding to the mapping to be multiplied] 2 j Since the multiplication is a homomorphism, GF(2 32 )⇔(Z / 2Z) 32 By identifying it with j The multiplication is done by the matrix P j ∈M(Z / 2Z,32×32). [Matrix P j How to calculate Put the power part of α as equation (14), and the k included in the coefficient of each term is k=31*2 j , …, k =1*2 j , k=0*2 j When arranged vertically, it becomes equation (33). The result of applying the result of Equation (33) to Equation (32) is Equation (34). matrix P j is the right part of equation (34). In addition, 2 j Since the exponential map is a composition of j square maps, the matrix P j =(P1) j You can also ask for:

[0074]

number

[0075]

number

[0076]

number

[0077]

number

[0078] [Matrix P j exponentiation by In the data restoration system, the exponentiation process using Equation (34) is implemented in the exponentiation means of a computer. In the data recovery system, the exponentiation means for performing the exponentiation process includes a decomposition means for converting the number of exponentiations n of the formula (16) given as the exponentiation target a into the formula (35), and a coefficient n in binary notation of the number of exponentiations n based on the result obtained by the decomposition means. j n out of (0≦j≦31) j For j where is "1", in each term of formula (15), α of formula (14) 31 From α 0 The coefficient of k=31*2 j ,k=30*2 j , …,k=1*2 j ,k=0*2 j The fitted e(31*2 j ,31) …e(31*2 j ,0) …e(0*2j ,31) …e(0*2 j ,0) is the matrix P j A data table that holds the value of (0≦j≦31) and the coefficient n of the binary notation of the exponentiation number n j n out of (0≦j≦31) j All a^2 for which is "1" j The coefficients of each term of the exponent a (Equation (16)) are 31 ...a1a0) and the matrix P whose elements are the values ​​stored in the data table j The product of (a 31 …a1a0)P j and a shaping means for outputting all a^2 output from the shaping means. j and a multiplication means for multiplying and outputting the result. matrix P j GF(2 32 The components of (the right part of equation (34)) are stored in a storage device as a data table for exponentiation by a PC. matrix P j Each data table is 32 bits x 32 = 4 bytes x 32 = 128 bytes. Also, 0 ≤ j ≤ 31, and matrices P0, P1, ..., P 30 ,P 31 Therefore, the total size of the data table used for the exponentiation calculation is 128 Bytes x 32 = 4096 Bytes.

[0079]

number

[0080]

number

[0081]

number

[0082]

number

[0083] [How to calculate general exponents] The algorithm for making a PC perform general exponential power calculations is as follows: In this case, the following input 1: exponentiation target (Equation (36)) and input 2: number of times of exponentiation (Equation (35)) are given to the algorithm, and the output is: a n Here, "n j (0≦j≦31)" is a binary coefficient and is either 1 or 0.

[0084]

number

[0085]

number

[0086] Output: a n Local variable 1: power_two(a^2 j (Enter the value as needed) Here, "^" is exponentiation. Local variable 2: power (eventually a n (return value will be Local variable 3: j (loop variable)

[0087] If(a==0) { return 0; } / / Here a≠0 power=1; For(j=0;j≦31;j++) { if(n j ==1) { power_two= (a 31 , …, a1, a0)P j ; / / power_two⇔a^2 j power = power × power_two; / / × is GF(2 32 ) component-by-component multiplication } / / By induction on j, the power immediately after the jth loop is completed is given by formula (25) } / / for / / The power immediately after the 31st loop is a n . Return power;

[0088]

number

[0089] <Hardware configuration> The example shown in Figure 1 is a distributed data storage system that includes a personal computer (PC) 5 that performs various processes and calculations on data, internal storage such as an HDD or memory installed in the PC 5, external storage such as a hard disk (HDD), solid state drive (SSD), USB memory, optical disk, or SD card, and a controller 3 and data communication network (hereinafter referred to as "network 4") that mediate mutual communication between the PC 5, the internal storage, and the external storage. This storage system includes an original data storage 1 for distributing and storing original data across multiple original data units, and a redundant data storage 2 for distributing and storing redundant data units for each of the multiple original data units in the original data storage 1.

[0090] <Functional configuration> The controller 3 is configured as a so-called computer system. When a data restoration program is run, the controller 3 functions as an original unit creation means for distributing and recording data input as a storage target in the external storage and internal storage as original data units in original data storage 1, a redundant unit creation means for creating each redundant data unit from each original data unit of the original data recorded in original data storage 1 and distributing and recording the redundant data units individually in redundant data storage 2, an original unit restoration means for restoring a damaged original data unit from an undamaged original data unit among the original data units recorded in original data storage 1 and an undamaged redundant data unit among the redundant data units recorded in redundant data storage 2 and recording the damaged original data unit in the original data storage 1, and a redundant unit restoration means for restoring a damaged redundant data unit from all original data units including the original data unit restored in original data storage 1 and recording the damaged redundant data unit in redundant data storage 2. Note that the creation of original data units by the original unit creation means is shown as an example, and any method may be used for creating original data units.

[0091] <Contents of the original data unit> The original unit creation means distributes and records input data to be saved into M (an integer of 2 or more) original data units in one or more original data storages 1 (original unit creation process). Each original data unit has N (an integer of 1 or more) components. That is, M is the number of original data units, and N is the number of components per data unit. Here, the i-th original data unit D[i] is a data disk or a data file in which normal original data is stored. The original data unit D[i] is a vector of GF(2 32 ) is a horizontal vector with N components. Therefore, we consider the sequences to be D[i]={D[i][0],...,D[i][j],...,D[i][N-1]}. When M elements D[0] to D[N-1] of the original data unit are arranged vertically, the matrix of formula (37) is obtained. where D[i][j]∈GF(2 32 ), (0≦i≦M-1 and 0≦j≦N-1). That is, D[i]∈[GF(2 32 ) N is.

[0092]

number

[0093] <Creating redundant data units> The redundant unit creation means creates R (an integer equal to or greater than 2) redundant data units for repairing damage by performing calculations of a redundant unit creation algorithm on each of the original data units recorded by the original unit creation means, and distributes and records these in one or more redundant data storages 2 (redundant unit creation process). Each redundant data unit has N (an integer equal to or greater than 1) components. That is, R is the number of redundant data units, and N is the number of components per data unit.

[0094] Here, the h-th redundant data unit Q[h] (0≦h≦R−1) is a data disk or a data file in which normal redundant data is stored. The redundant data unit Q[h] is a vector of N GF(2 32 ) is a horizontal vector with components Therefore, we consider that Q[h] is arranged as Q[h][0], ..., Q[h][j], ..., Q [h] [N-1]}. When R components Q[0] to Q[N-1] of the redundant data unit are arranged vertically, the matrix of Equation (38) is obtained. where Q[h][j]∈GF(2 32 ), (0≦h≦R-1 and 0≦j≦N-1). That is, Q[h]∈[GF(2 32 ) N is.

[0095]

number

[0096] The redundant unit creation means creates a number list "0, 1, ..., R-1" of R (R ≥ 2) redundant data units (Q[0], ..., Q[R-1]), creates a number list "0, 1, ..., M-1" of M (M ≥ 2) original data units (D[0], ..., D[M-1]), and creates a matrix (Equation (39)) in which the products of the elements of the number list of the redundant data units and the elements of the number list of the original data units are arranged vertically and horizontally.

[0097]

number

[0098] Next, a matrix is ​​created in which the elements of the matrix are powers of α (32-bit Galois field) arranged vertically and horizontally, and a group of redundant data units is generated by deriving the product (Equation (3)) with the group of original data units (redundant unit creation algorithm). For example, when the number of original disks M=7 and the number of redundant disks R=6, the formula (40) is obtained.

[0099]

number

[0100]

number

[0101] <Restoring the original data unit> When the original data unit is damaged, the original unit restoration means restores the damaged original data units D[x[0]],..., D[x[L-1]] from M-L non-damaged original data units D[y[0]],..., D[y[M-L-1]] and L redundant data units Q[a[0]],..., Q[a[L-1]] extracted from R-K non-damaged redundant data units, and records them in the original data storage 1. The original unit restoration means includes a unit extraction means for detecting damaged redundant data units and a unit restoration means for restoring damaged original data units from non-damaged original data units and non-damaged redundant data units, and sequentially performs unit extraction processing and original unit restoration processing (see FIGS. 2 and 3).

[0102] [Unit Extraction Processing] The unit extraction means creates a number list list_a that detects and identifies non-damaged redundant data units, and creates a number list list_x that detects and identifies damaged original data units. Next, the unit extraction means creates a number list list_y of non-damaged original data units excluding the elements of the number list list_x from the number list {0, 1,..., M-1} of original data units. And when L (the number of damaged original data units) ≤ R (the number of redundant data units) - K (the number of damaged redundant data units), L redundant data units are extracted from the elements of the number list list_a in ascending order of numbers, and an extraction list use_list_a consisting of those numbers is created (see FIG. 2(b)). However, if R-K < L, the creation of the extraction list use_list_a and the original unit restoration processing are abandoned.

[0103] Here The number list of non-damaged redundant data units is list_a = {a[0], a[1], …, a[R-K-2], a[R-K-1]} The number list of damaged redundant data units is list_b = {b[0], b[1], …, b[0], b[K - 1]} The list of numbers of the damaged original data units is list_x = {x[0], x[1], …, x[L - 2], x[L - 1]}, and The list of numbers of the non - damaged original data units is list_y = {y[0], y[1], …, y[M - L - 2], y[M - L - 1]}, and The extraction list of L non - damaged redundant data units is Assume Use_list_a = {a[0], a[1], …, a[L - 2], a[L - 1]}. If R - K < L, the extraction list Use_list_a is an empty list.

[0104] [Original unit repair process] Subsequently, the original unit repair means extracts L rows belonging to the extraction list use_list_a from the formula (3) at the time of creating the redundant data units to create a matrix, and separates them according to whether there is damage as in formula (41) and formula (42). At that time, the original unit repair means creates a matrix Mat(list_x, use_list_a) for the damaged original data units from the list of numbers of the damaged original data units list_x created as above and the extraction list use_list_a, and creates a matrix Mat(list_y, use_list_a) for the non - damaged original data units from the list of numbers of the non - damaged original data units list_y and the extraction list use_list_a (see formula (42)).

[0105] [Number]

[0106] [Number]

[0107] Furthermore, the original unit repair means calculates an inverse matrix Mat(list_x, use_list_a) of the matrix Mat(list_x, use_list_a) for the corrupted original data units. -1 and derive the product of this with the matrix Mat(list_y, use_list_a) for the uncorrupted original data units (see Equation (43)). In addition, the inverse matrix of Mat(list_x,use_list_a) is Mat(list_x,use_list_a) -1 is 2 -32 There is a probability that it does not exist.

[0108]

number

[0109] Next, the inverse matrix Mat(list_x,use_list_a) of the matrix Mat(list_x,use_list_a) for the corrupted original data units is calculated. -1 and a group (matrix) of L uncorrupted redundant data units, and the matrix Mat(list_x, use_list_a) -1 Derive the product of Mat(list_y, use_list_a) and the set (matrix) of ML uncorrupted original data units. Finally, the sum of the two matrices derived as their respective products is derived, and the restoration of the original data unit is completed (see Equation (4)).

[0110]

number

[0111] For example, if M=7, R=6, L=3, and the number list List_x:=[1,4,6], number list list_y:=[0,2,3,5], and extraction list use_list_a:=[2,3,5], then by extracting only the rows belonging to the extraction list use_list_a from formula (40) for creating redundant data and separating them based on whether they are damaged or not, the damaged original data units D[1], D[4], and D[6] can be restored by sequentially going through the processes from formula (44) to formula (47).

[0112]

number

[0113]

number

[0114]

number

[0115]

number

[0116] [Redundant unit restoration process] The redundant unit restoration means restores corrupted redundant data units Q[b[0]], ..., Q[b[K-1]] from all original data units D[0], ..., D[M-1], including the original data units restored by the original unit repair means. When a redundant data unit is damaged, the redundant unit restoration means restores the damaged redundant data units Q[b[0]], ..., Q[b[K-1]] by performing a redundant unit restoration algorithm on all original data units D[0], ..., D[M-1], including the original data units restored by the original unit restoration means, and records them in the redundant data storage 2. The redundant data units Q[b[0]], ..., Q[b[K-1]] are restored as shown in equation (5).

[0117]

number

[0118] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a data recovery system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Here, it is assumed that the original data is divided into seven original data files (original data units) D[0], D[1], D[2], D[3], D[4], D[5], and D[6] as shown in Tables 1 and 2 by the original unit creation means and recorded in seven original data storages 1 (see Figure 1 and Table 1). In this embodiment, each original data file D[i] stores 48 characters (UINT_DATA_SIZE×ONE_BLOCK_SIZE×BLOCK_COUNT=4×4×3) of alphabets (1 character = 1 byte) (see Tables 1 and 2). In the data restoration method and data restoration system, when calculations are performed using a 32-bit (4-character) Galois field, N=12=48÷4 BYTE.

[0119] [Table 1]

[0120] [Table 2]

[0121] [Table 3]

[0122] [Table 4]

[0123] [Table 5]

[0124]

Table 6

[0125]

Table 7

[0126]

Table 8

[0127] These examples show an example of the processing in the data restoration system (see FIGS. 3 and Tables 2 to 8). In this data restoration system, when the number (R - K) of intact redundant data files (redundant data units) is greater than or equal to the number L of damaged original data files (R - K ≥ L), the original data files can be restored. When the number (R - K) of intact redundant data files is less than the number L of damaged original data files (R - K < L), restoration is impossible.

[0128] [CaseA] In CaseA, since the number 4 (R - K) of intact redundant data files exceeds the number 3 (L) of damaged original data files (R - K > L), subsequently, the original unit restoration process is performed. At this time, in the unit extraction process, the unit extraction means detects intact redundant data units and creates a number list list_a = [1, 2, 4, 5] to identify them, and detects damaged original data files and creates a number list list_x = [1, 3, 4] to identify them.

[0129] Next, the unit extraction means creates a number list list_y=[0,2,5,6] of uncorrupted original data files by excluding elements of the number list list_x=[1,3,4] of corrupted original data files from the number list {0,1, ...,6} of original data files, and also creates a number list list_b=[0,3,6] of corrupted redundant data files by excluding elements of the number list list_a=[1,2,4,5] of uncorrupted redundant data files from the number list {0,1, ...,6} of redundant data files. Furthermore, the unit extraction means deletes the original data files D[1], D[3], D[4] associated with the number list list_x=[1,3,4] of the damaged original data files from the original data files D[0], ..., D[6], and deletes the redundant data files Q[0], Q[3], Q[6] associated with the number list list_b=[0,3,6] of the damaged redundant data files from the redundant data files Q[0], ..., Q[6] (see Figure 5).

[0130] In the original unit repair process, the original unit repair means reads the original data files D[0], D[2], D[5], D[6] associated with the number list list_y=[0,2,5,6] of uncorrupted original data files and the redundant data files Q[1], Q[2], Q[4] associated with the number list use_list_a=[1,2,4] of uncorrupted redundant files into the original unit repair algorithm of the original files (the redundant data file Q[5] associated with the number list list_a\use_list_a=[5] of uncorrupted redundant files is left over), and restores the original data files D[1], D[3], D[4] associated with the number list list_x=[1,3,4] of the corrupted original data files (see Figure 6).

[0131] Finally, in the redundant unit restoration process for the damaged redundant data file, the redundant unit restoration means reads all original data files D[0], ..., D[6], including original data files D[1], D[3], D[4] associated with the restored original data file number list list_x=[1,3,4], into a redundant unit restoration algorithm (redundant unit creation algorithm), and restores redundant data files Q[0], Q[3], Q[6] associated with the damaged redundant data file number list list_b=[0,3,6].

[0132] [Case B] Case B shows an example of processing in a data recovery system in which all of the original data files D[0], ..., D[6] are corrupted, but none of the redundant data files Q[0], ..., Q[6] are corrupted (M=R=7) (see Figure 9). In this example, the number of uncorrupted redundant data files (7, RK) is the same as the number of corrupted original data files (7, L) (RK=L), so the original unit recovery process is then performed. In this case, the unit extraction means detects uncorrupted redundant data files in the unit extraction process and creates a number list list_a=[0,1,2,3,4,5,6] to identify them, and detects corrupted original data files and creates a number list list_x=[0,1,2,3,4,5,6] to identify them.

[0133] Next, the unit extraction means creates a list of numbers list_y=[] (empty string) of uncorrupted original data files by excluding elements related to the number list list_x=[0,1,2,3,4,5,6] from the list of numbers {0,1, ...,6} of original data files, and also creates a list of numbers list_b=[] (empty string) of corrupted redundant data files along with the list of numbers list_a=[0,1,2,3,4,5,6] of uncorrupted redundant data files from the list of numbers {0,1, ...,6} of redundant data files. At this time, since RK≧7, the extraction list use_list_a can be defined as [0,1,2,3,4,5,6].

[0134] In the unit extraction process, the unit extraction means deletes the original data files D[0], …, D[6] corresponding to the list of numbers of the damaged original data files list_x = [0, 1, 2, 3, 4, 5, 6] from the original data files D[0], …, D[6], and deletes the redundant data files corresponding to the list of numbers of the damaged redundant data files list_b = [] from the redundant data files Q[0], …, Q[6] (see Fig. 8). In the case of this example, there are no damaged redundant data files, and there are no redundant data files to be deleted.

[0135] In the original unit repair process, the original unit repair means reads the original data files corresponding to the list of numbers of the non-damaged original data files list_y = [] and the redundant data files Q[0], …, Q[6] corresponding to the list of numbers of the non-damaged redundant data files use_list_a = [0, 1, 2, 3, 4, 5, 6] into the original unit repair algorithm (there is no remainder in the redundant data files corresponding to the list of numbers of the redundant data files list_a = [0, 1, 2, 3, 4, 5, 6]), and restores the original data files D[0], …, D[6] corresponding to the list of numbers of the damaged original data files list_x = [0, 1, 2, 3, 4, 5, 6] (see Fig. 9).

[0136] Finally, in the redundant unit restoration process of the damaged redundant data files, since there are no numbers of redundant data files in the list of numbers of the damaged redundant data files list_b = [], the process of Case B ends.

[0137] [Case C] This example shows an example of the processing in a data restoration system when the number of non-damaged redundant data files (R - K) is equal to the number of damaged original data files L ((R - K) = L < M, and there is no remainder in the redundant data files corresponding to the list of numbers of the non-damaged redundant data files list_a = [1, 2, 4]) (see Fig. 11).

[0138] In this case, the unit extraction means detects redundant data files that are not corrupted in the unit extraction process and creates a number list list_a=[1,2,4] that identifies them, and detects corrupted original data files and creates a number list list_x=[1,3,4] that identifies them.

[0139] Next, the unit extraction means creates a number list list_y=[0,2,5,6] of uncorrupted original data files by excluding elements related to number list list_x=[1,3,4] from the number list {0,1, ...,6} of original data files, and also creates a number list list_b=[0,3,5,6] of corrupted redundant data files by excluding elements related to number list list_a=[1,2,4] of uncorrupted redundant data files from the number list {0,1, ...,6} of redundant data files. At this time, since RK≧L, the extraction list use_list_a can be defined as [1,2,4].

[0140] Furthermore, in the unit extraction process, the unit extraction means deletes the original data files D[1], D[3], D[4] associated with the number list list_x=[1,3,4] of the damaged original data files from the original data files D[0], ..., D[6], and deletes the redundant data files Q[0], Q[3], Q[5], Q[6] associated with the number list list_b=[0,3,5,6] of the damaged redundant data files from the redundant data files Q[0], ..., Q[6] (see Figure 10).

[0141] The original unit repair means, in the original unit repair process, reads the original data files D[0], D[2], D[5], D[6] corresponding to the list of numbers of non-damaged original data files list_y = [0, 2, 5, 6], and the redundant data files Q[1], Q[2], Q[4] corresponding to the list of numbers of non-damaged redundant data files use_list_a = [1, 2, 4] into the original unit repair algorithm (there is no remainder in the elements corresponding to the list of numbers of redundant data files list_a = [1, 2, 4]), and restores the original data files D[1], D[3], D[4] corresponding to the list of numbers of damaged original data files list_x = [1, 3, 4] (see Figure 11).

[0142] Finally, the redundant unit restoration means, in the redundant unit restoration process of the damaged redundant data files, reads all the original data files including the original data files D[1], D[3], D[4] corresponding to the list of numbers of restored original data files list_x = [1, 3, 4] into the redundant unit restoration algorithm (redundant unit creation algorithm), and restores the redundant data files Q[0], Q[3], Q[5], Q[6] corresponding to the list of numbers of damaged redundant data files list_b = [0, 3, 5, 6] (see Figure 12).

[0143] [CaseD] This example shows an example of the processing in a data restoration system when the number of non-damaged redundant data files (R - K) is less than the number of damaged original data files L (R - K < L, and the elements corresponding to the list of numbers of redundant data files list_a = [1, 4] are insufficient) (see Figure 14). At this time, the unit extraction means, in the unit extraction process, detects non-damaged redundant data files and creates a list of numbers list_a = [1, 4] to identify them, and detects damaged original data files and creates a list of numbers list_x = [1, 3, 4] to identify them.

[0144] Next, the unit extraction means creates an intact original data file number list list_y = [0, 2, 5, 6] by removing the elements related to the number list list_x = [1, 3, 4] from the original data file number list {0, 1, …, 6}, and creates a damaged redundant data file number list list_b = [0, 2, 3, 5, 6] by removing the elements related to the intact redundant data file number list list_a = [1, 4] from the redundant data file number list {0, 1, …, 6}. At this time, since R - K < L, the extraction list use_list_a := [] can be defined.

[0145] Furthermore, the unit extraction means deletes the original data files D[1], D[3], D[4] related to the damaged original data file number list list_x = [1, 3, 4] from the original data files D[0], …, D[6], and deletes the redundant data files Q[0], Q[2], Q[3], Q[5], Q[6] related to the damaged redundant data file number list list_b = [0, 2, 3, 5, 6] from the redundant data files Q[0], …, Q[6] (see Fig. 13).

[0146] With the extraction list use_list_a := [] defined, the original unit repair means terminates the original unit repair process without loading the original data files D[0], D[2], D[5], D[?6] related to the intact original data file number list list_y = [0, 2, 5, 6] and the redundant data files Q[1], Q[4] related to the intact redundant file number list list_a = [1, 4] into the original unit repair algorithm of the original file (see Fig. 14(b)).

[0147] Finally, in the redundant unit restoration process of the damaged redundant data files, since the restored original files D[1], D[3], D[4] do not exist (remain damaged), the redundant data files Q[0], Q[2], Q[3], Q[5], Q[6] related to the damaged redundant data file number list list_b = [0, 2, 3, 5, 6] are not restored.

[0148] The original unit restoration means sequentially performs the unit extraction process and the original unit repair process as described above, but if it determines that restoration is impossible, it stops the original unit restoration process including the original unit repair process and does not perform the redundant unit restoration process. The determination that restoration is impossible due to the occurrence of cases such as Cases A to D above is also output when an inverse matrix does not exist for a matrix created in a 32-bit Galois field. In such a case, the probability that the original unit restoration means outputs a message that restoration is impossible (Mat(list_x, use_list_a) does not have an inverse matrix) (the original unit restoration algorithm stops) is approximately 2 -32 If the above method is performed in 8-bit units, the probability of outputting a judgment that restoration is impossible is approximately 2 -8 If the above method is performed in 16-bit units, the probability of outputting a judgment that recovery is impossible is approximately 2 -16 This becomes: At present, it has been confirmed that if M≦14 and R≦14, the inverse matrix always exists. [Explanation of symbols]

[0149] D Original data unit (file), Q Redundant data unit (file), 1. Original data storage, 2. Redundant data storage, 3 Controller, 4 Network, 5. Personal Computer (PC)

Claims

1. a redundant unit creating means for creating R (R is a natural number of 2 or more) redundant data units Q[0] to Q[R-1] for restoring M (M is a natural number of 2 or more) original data units D[0] to D[M-1]; a data restoration system including: an original unit restoration means for restoring the damaged L original data units from R-K (a natural number) redundant data units that are not damaged when L (a natural number) of the original data units are damaged; The redundant unit creation means creates R redundant data units Q[0] to Q[R-1] from M original data units D[0] to D[M-1] through the calculation process of Equation (3), The original unit restoration means extracts L uncorrupted redundant data units to be used to restore the original data unit when L≦R−K, and restores the original data unit from the extracted L redundant data units Q[a[0]], ..., Q[a[L−1]] and the M−L uncorrupted original data units D[y[0]], ..., D[y[M−L−1]] through the calculation process of equation (4). A data restoration system characterized by the above. [Equation 1] [Equation 2]

2. The data recovery system according to claim 1, further comprising a redundant unit recovery means for recovering corrupted redundant data units Q[b[0]], ..., Q[b[K-1]] from all original data units [0], ..., D[M-1] including the recovered original data units through the calculation process of equation (5) after the original data units are recovered by the original unit recovery means. [Equation 3]

3. In the above calculation process, f is calculated for the 32nd degree polynomial of Z / 2Z coefficient (Equation (6)). 32 a multiplication means for multiplying an arbitrary element a (formula (8)) and an element b (formula (9)) of a 32-bit Galois field generated by α, where α is a solution of (x)=0, to obtain a product ab=c; [Equation 4] [Equation 5] [Equation 6] [Equation 7] The multiplication means a calculation means for obtaining a 62nd degree polynomial C(x) (formula (12)) which is the product of A(x) (formula (10)) and B(x) (formula (11)) in which α of the element a and the element b is replaced with a variable x; [Equation 8] [Equation 9] [Equation 10] Regarding the power part of α in C(α) obtained by substituting α for x in C(x) (Equation (12)), α k degree conversion means for converting the polynomial into a 31st degree polynomial (Equation (14)) using coefficients e(k,j) (0≦k≦62, 0≦j≦31) that reduce the polynomial to a 31st degree polynomial by using Equation (13) when the exponent of [0011] [0012] The α used in the multiplication process k a data table that stores the values ​​of a matrix whose components are the coefficients (e(k,31) e(k,30) ... e(k,1) e(k,0)); The product output from the calculation means is used as the coefficient (c 62 … c 32 c 31 … c 1 c 0 3. The data recovery system according to claim 1, further comprising a shaping means for outputting a product of elements of the matrix stored in said data table and the matrix stored in said data table.

4. In the above calculation process, f is calculated for the 32nd degree polynomial of Z / 2Z coefficient (Equation (6)). 32 Let the solution of (x)=0 be α, and for any element a (Equation (8)) of the 32-bit Galois field generated by α, which is Equation (7), ｎ (0≦n≦2 32 -1), [0013] [0014] [Equation 15] The exponentiation means Decomposition means for converting the number of times n to be exponentiated of the exponentiation target a into the form of Equation (35); [0016] Based on the result obtained by the decomposition means, the coefficient n of the exponentiation number n in binary notation is calculated. j (0≦j≦31) j For j where is "1", in each term of Equation (15), α 31 From α 0 The coefficient of k=31*2 j , k=30*2 j , …,k=1*2 j ,k=0*2 j The fitted e(31*2 j ,31)…e(31*2 j ,0)…e(0*2 j ,31)…e(0*2 j ,0) is the matrix P j A data table that holds values ​​of (0≦j≦31), [Equation 17] [Equation 18] Coefficient n of the exponentiation number n in binary notation j (0≦j≦31) j All a^2 for which is "1" j The coefficients of each term of the exponent a (Equation (16)) (a 31 …a 1 a 0 ) and a matrix P whose elements are the values ​​stored in the data table j The product of (a 31 …a 1 a 0 ) P j A formatting means for outputting the data in the form of [Equation 19] All a^2 output from the shaping means j 3. The data recovery system according to claim 1, further comprising a multiplication means for multiplying the signal by the signal and outputting the result.

5. In the above calculation process, f is calculated for the 32nd degree polynomial of Z / 2Z coefficient (Equation (6)). 32 A multiplication means for multiplying an arbitrary element a (formula (8)) and an element b (formula (9)) of a 32-bit Galois field generated by α, where α is a solution of (x)=0, to obtain a product ab=c; In the above calculation process, f is calculated for the 32nd degree polynomial of Z / 2Z coefficient (Equation (6)). 32 Let the solution of (x)=0 be α, and for any element a (Equation (8)) of the 32-bit Galois field generated by α, which is Equation (7), ｎ (0≦n≦2 32 and exponentiation means for deriving [Equation 20] [0000] [Equation 22] [Equation 23] The multiplication means a calculation means for obtaining a 62nd degree polynomial C(x) (formula (12)) which is the product of A(x) (formula (10)) and B(x) (formula (11)) in which α of the element a and the element b is replaced with a variable x; [0000] [Equation 25] [Equation 26] Regarding the power part of α in C(α) obtained by substituting α for x in C(x) (Equation (12)), α k degree conversion means for converting the polynomial into a 31st degree polynomial (Equation (14)) using coefficients e(k,j) (0≦k≦62, 0≦j≦31) that reduce the polynomial to a 31st degree polynomial by using Equation (13) when the exponent of [0000] [0000] The α used in the multiplication process k a data table that stores the values ​​of a matrix whose components are the coefficients (e(k,31) e(k,30) ... e(k,1) e(k,0)); The product output from the calculation means is used as the coefficient (c 62 … c 32 c 31 … c 1 c 0 ) and a matrix stored in the data table in the form of a product of the components thereof, The exponentiation means Decomposition means for converting the number of times n to be exponentiated of the exponentiation target a into the form of Equation (35); [0000] Based on the result obtained by the decomposition means, the coefficient n of the exponentiation number n in binary notation is calculated. j (0≦j≦31) j For j where is "1", in each term of Equation (15), α 31 From α 0 The coefficient of k=31*2 j , k=30*2 j , …,k=1*2 j ,k=0*2 j The fitted e(31*2 j ,31)…e(31*2 j ,0)…e(0*2 j ,31)…e(0*2 j ,0) is the matrix P j A data table that holds values ​​of (0≦j≦31), [Equation 30] [Equation 31] Coefficient n of the exponentiation number n in binary notation j (0≦j≦31) j All a^2 for which is "1" j The coefficients of each term of the exponent a (Equation (16)) (a 31 …a 1 a 0 ) and a matrix P whose elements are the values ​​stored in the data table j The product of (a 31 …a 1 a 0 ) P j A formatting means for outputting the data in the form of [Equation 32] All a^2 output from the shaping means j 3. The data recovery system according to claim 1, further comprising a multiplication means for multiplying the signal by the signal and outputting the result.

Images