Data recovery methods, apparatus, equipment and storage media

Abstract

The application provides a data repairing method, device, equipment and storage medium. The method can comprise: acquiring a plurality of data blocks, and grouping the plurality of data blocks to obtain a plurality of local encoding groups; the local encoding group comprises at least one data block; generating a global check block based on the plurality of data blocks, and generating a local check block corresponding to each local encoding group based on the global check block and the data blocks contained in each local encoding group; and repairing data based on the local check block and the global check block. Thus, when repairing the global check block, all data blocks do not need to be read, the repairing traffic required for repairing the global check block is reduced, and the repairing cost is further reduced.

Description

Technical Field

[0001] This application relates to computer technology, specifically to a data recovery method, apparatus, device, and storage medium. Background Technology

[0002] Locally Repairable Codes (LRC) is an encoding method that adds local parity blocks on top of global parity blocks. Its core idea is to generate a global parity block based on all data and to divide all data into multiple local coding groups, generating local reconstruction parity blocks based on the data blocks within the local coding groups.

[0003] When there is only a single data block that needs to be reconstructed within a local coding group, the system can directly recover the lost data block using other data blocks and local check blocks within that local coding group.

[0004] LRC is increasingly being used in various storage systems. LRC encoding can significantly reduce the repair traffic required when a single data block is repaired. However, when the global parity block is lost, the system still needs to read all the data to complete the repair, resulting in higher repair traffic and higher repair costs. Summary of the Invention

[0005] In view of this, this application proposes at least one data repair method. The method may include: acquiring multiple data blocks, and grouping the multiple data blocks into multiple local coding groups; each local coding group includes at least one of the data blocks; generating a global verification block based on the multiple data blocks, and generating local verification blocks corresponding to each local coding group based on the global verification block and the data blocks contained in each local coding group; and performing data repair based on the local verification blocks and the global verification block.

[0006] In some embodiments, generating a global verification block based on the plurality of data blocks includes: obtaining a first coefficient preset for each of the data blocks; and generating a global verification block based on the first coefficient and the plurality of data blocks.

[0007] In some embodiments, generating a local check block corresponding to each local coding group based on the global check block and the data blocks contained in each local coding group includes: obtaining a second coefficient corresponding to each local coding group, wherein the second coefficient indicates preset coefficient information of the data blocks in the local coding group and the global check block; and generating a local check block corresponding to each local coding group based on the second coefficient, the data blocks in the local coding group, and the global check block.

[0008] In some embodiments, before generating a global check block based on the first coefficient and the plurality of data blocks, and before generating a local check block corresponding to each of the local coding groups based on the second coefficient, the data blocks within the local coding groups, and the global check block, the method further includes: generating a first coding row in the coding matrix corresponding to the global check block based on the first coefficient; the first coding row being used to perform a dot product with the plurality of data blocks to obtain the global check block; generating a second coding row corresponding to each of the local coding groups based on the first coefficient and the second coefficient corresponding to each of the local coding groups, such that the second coding row incorporates the first coefficient required to generate the global check block; the second coding row being used to perform a dot product with the plurality of data blocks to obtain the local check block; and generating a coding matrix based on the first coding row and the second coding row.

[0009] In some embodiments, the second coefficient includes a first sub-coefficient preset for the global check block and a second sub-coefficient preset for the data blocks within the local coding group; generating a second coding line corresponding to each local coding group based on the first coefficient and the second coefficient corresponding to each local coding group includes, for each local coding group: multiplying each first coefficient by the first sub-coefficient to obtain a third coefficient for the plurality of data blocks; adding the third coefficient of the data blocks within the local coding group to the second sub-coefficient to update the third coefficient of the data blocks within the local coding group; and generating the second coding line based on the current third coefficient.

[0010] In some embodiments, generating a global check block based on the first coefficient and the plurality of data blocks, and generating a local check block corresponding to each of the local coding groups based on the second coefficient, the data blocks within the local coding group, and the global check block, includes: obtaining the coding matrix; generating the global check block based on the dot product result of the first coding row in the coding matrix and the plurality of data blocks; and generating a local check block corresponding to each of the local coding groups based on the dot product result of each second coding row in the coding matrix and the plurality of data blocks.

[0011] In some embodiments, the data repair based on the local check block and the global check block includes at least one of the following: when the global check block needs repair, generating the global check block to be repaired based on the data blocks within the local check block generated from the global check block to be repaired and the corresponding local check blocks, to complete the data repair; when any data block among the plurality of data blocks needs repair, generating the data block to be repaired based on other intact data blocks within the local coding group to which the data block to be repaired belongs and the corresponding local check blocks, to complete the data repair; when the local check block needs repair, generating the local check block to be repaired based on the data blocks within the local check block to be repaired and the global check block, to complete the data repair.

[0012] This application also proposes a data repair apparatus, comprising: an acquisition module for acquiring multiple data blocks and multiple local coding groups obtained by grouping the multiple data blocks; each local coding group includes at least one of the data blocks; a generation module for generating a global verification block based on the multiple data blocks, and generating a local verification block corresponding to each local coding group based on the global verification block and the data blocks contained in each local coding group; and a repair module for performing data repair based on the local verification blocks and the global verification blocks.

[0013] This application also proposes an electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor implements a data repair method as shown in any of the foregoing embodiments by executing the executable instructions.

[0014] This application also proposes a computer-readable storage medium storing a computer program for causing a processor to execute a data repair method as shown in any of the foregoing embodiments.

[0015] In the aforementioned scheme, by allowing the global check block to participate in the generation of the local check copy, compared with LRC-related technologies, the local check block can contain information of the global check block. Thus, during the data repair process, if it is necessary to repair the global check block, the repair can be performed based on the local check block and the data blocks within the local coding group, without having to read all the data blocks. This reduces the repair traffic required to repair the global check block, thereby reducing the repair cost.

[0016] It should be understood that the general description above and the detailed description below are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in one or more embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in one or more embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart illustrating a data repair method according to an embodiment of this application;

[0019] Figure 2 This is a schematic flowchart illustrating a method for generating an encoding matrix according to an embodiment of this application;

[0020] Figure 3 This is a flowchart illustrating a method for generating a second encoded line according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram illustrating a data storage scenario as shown in an embodiment of this application;

[0022] Figure 5 This is a schematic flowchart illustrating a method for storing data according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the structure of a data repair device according to an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of devices and methods consistent with some aspects of this application as detailed in the appended claims.

[0026] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items. It should also be understood that the word “if” as used herein, depending on the context, can be interpreted as “when,” “in response to a determination,” or “when…”.

[0027] Based on this, this application proposes a data repair method. The method may include: acquiring multiple data blocks, and grouping the multiple data blocks into multiple local coding groups; each local coding group includes at least one of the data blocks; generating a global verification block based on the multiple data blocks, and generating local verification blocks corresponding to each local coding group according to the global verification block and the data blocks contained in each local coding group; and performing data repair based on the local verification blocks and the global verification block.

[0028] In the aforementioned scheme, by allowing the global check block to participate in the generation of the local check copy, compared with LRC-related technologies, the local check block can contain information of the global check block. Thus, during the data repair process, if it is necessary to repair the global check block, the repair can be performed based on the local check block and the data blocks within the local coding group, without having to read all the data blocks. This reduces the repair traffic required to repair the global check block, thereby reducing the repair cost.

[0029] The following description, in conjunction with the accompanying drawings, illustrates the embodiments. Please refer to the attached figures. Figure 1 , Figure 1 This is a schematic flowchart illustrating a data repair method according to an embodiment of this application.

[0030] Figure 1 The data repair method shown can be applied to electronic devices. The electronic device can execute the method by incorporating software logic corresponding to the data repair method. The type of electronic device can be a laptop, computer, server, mobile phone, PDA, etc. This application does not specifically limit the type of electronic device. The electronic device can also be a client device or a server device, without particular limitation.

[0031] like Figure 1 As shown, the method may include steps S102-S106. Unless otherwise specified, this application does not specifically limit the order in which these steps are performed.

[0032] In step S102, multiple data blocks are acquired, and multiple local coding groups are obtained by grouping the multiple data blocks.

[0033] The data blocks refer to blocks of data to be stored. In data storage scenarios, data to be stored is divided into multiple contiguous data blocks before storage. LRC (Local Responsive Code) is used as a coding fault-tolerance technique during the data block storage process.

[0034] During encoding, these data blocks are divided into multiple local encoding groups. This application does not limit the grouping method.

[0035] In some methods, multiple data blocks can be evenly divided into local coding groups based on a preset number of local coding groups. For example, if the data to be stored is divided into 6 data blocks and the number of local coding groups is 2, then each local coding group can include 3 data blocks.

[0036] S104, based on the plurality of data blocks, generate a global check block, and based on the global check block and the data blocks contained in each of the local coding groups, generate a local check block corresponding to each of the local coding groups.

[0037] In LRC-related technologies, the global check block refers to a check block generated based on all data blocks, while the local check block refers to a check block generated based on data blocks within a local coding group. This application adopts the concepts from LRC-related technologies.

[0038] Unlike related technologies, the local check block in this application includes not only the data block information contained within the local coding group, but also relevant information about the global check block.

[0039] In some embodiments, in order to achieve the purpose of including the data block information contained in the local coding group and the relevant information of the global verification block in the local verification block, a global verification block can be generated first based on the multiple data blocks, and then a local verification block corresponding to each local coding group can be generated according to the global verification block and the data blocks contained in each local coding group.

[0040] Specifically, a first coefficient can be obtained for each of the data blocks; then, a global verification block is generated based on the first coefficient and the plurality of data blocks.

[0041] The first coefficient can be set according to requirements. The first coefficient can be obtained from the Cauchy matrix or the Vandermen matrix.

[0042] After obtaining the first coefficient, a global verification block can be generated by constructing a vector dot product. The vector dot product operation includes addition and multiplication. The addition is the addition defined in the finite field GF(2^8), i.e., the XOR operation; the multiplication can all be the multiplication defined in the finite field GF(2^8), i.e., polynomial multiplication.

[0043] Taking a data block count of 6 as an example, the number of first coefficients can also be 6. In this application, x1 to x6 and y1 to y6 can be used to represent two sets of first coefficients, A1 to A6 can be used to represent 6 data blocks, and P1 and P2 can be used to represent two global check blocks.

[0044] After obtaining the six first coefficients, in some methods, the six first coefficients can be combined into a horizontal quantity, the six data blocks can be combined into a column vector, and then the global check block can be obtained through dot product operation.

[0045] The calculation formula is as follows:

[0046]

[0047]

[0048] Based on formulas (1) and (2), a global verification block can be generated using the first coefficient and multiple data blocks.

[0049] After obtaining the global check block, a local check block can be generated based on the global check block.

[0050] Specifically, a second coefficient corresponding to each of the local coding groups can be obtained, the second coefficient indicating the preset coefficient information of the data block in the local coding group and the global check block; then, a local check block corresponding to each of the local coding groups is generated based on the second coefficient, the data block in the local coding group and the global check block.

[0051] The second coefficient can be set according to requirements. In some embodiments, the value of the second coefficient is not limited. For example, the first coefficient can be a value of 1, 2, 3, etc. In some embodiments, the second coefficient is a weighting coefficient, that is, the sum of all the second coefficients is 1.

[0052] The sum of the second coefficients can be obtained by constructing a vector dot product to generate a local check block.

[0053] Continuing with the previous example, the number of data blocks is 6, and these 6 data blocks are divided into 2 local coding groups. Local coding group 1 includes data blocks A1 to A3, and local coding group 2 includes data blocks A4 to A6. In this application, z1 to z4 and r1 to r4 can be used to represent the two sets of second coefficients, L1 can be used to represent the local parity block corresponding to local coding group 1, and L2 can be used to represent the local parity block corresponding to local coding group 2.

[0054] After obtaining the four second coefficients, in some methods, for each local coding group, the four first coefficients can be combined into a horizontal vector, and the three data blocks within that local coding group and the corresponding global parity block can be combined into a column vector. Then, the local parity block is obtained through dot product. It should be noted that, in the case of multiple global parity blocks, this application does not limit the combination method of the global parity block and the local coding group. For example, with two global parity blocks P1 and P2 and two local coding groups 1 and 2, the local parity block can be generated using either P1 and the data blocks within group 1, or it can be generated using P2 and the data blocks within group 1.

[0055] In this example, it is assumed that local check block L1 is generated using P1 and the data blocks in group 1, which is beneficial for generating local check block L2 using P2 and the data blocks in group 2.

[0056] The calculation formula is as follows:

[0057]

[0058]

[0059] Based on formulas (3) and (4), the second coefficient, the global check block, and the data blocks within the local coding group can be used to generate a local check block corresponding to the local coding group.

[0060] In some embodiments, an encoding matrix can be generated based on the first coefficient and the second coefficient, which facilitates correlation operations between the encoding matrix and multiple data blocks to obtain a global check block and a local check block.

[0061] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating a method for generating an encoding matrix according to an embodiment of this application. Figure 2 As shown, the method includes steps S202-S206. Unless otherwise specified, this application does not limit the order in which these steps are performed.

[0062] S202, Based on the first coefficient, generate the first coding row in the coding matrix corresponding to the global check block.

[0063] The first encoded line is used to perform a dot product with the plurality of data blocks to obtain the global verification block.

[0064] In this step, the first coefficient corresponding to each data block can be used as an element in the first encoding line to obtain the first encoding line.

[0065] S204, based on the first coefficient and the second coefficient corresponding to each of the local coding groups, generate a second coding line corresponding to each of the local coding groups, so that the second coding line incorporates the first coefficient required to generate the global check block.

[0066] The second encoding line is used to perform a dot product with the plurality of data blocks to obtain the local check block. By incorporating the first weight coefficient into the second encoding line, global check block information can be introduced when calculating the local check block based on the second encoding line, thus enabling the global check block to participate in the generation of the local check copy.

[0067] The following section explains the principle behind generating the second encoded line, using the previous examples as examples.

[0068] Expanding the aforementioned formula (3), we can obtain: L1=z1*A1+z2*A2+z3*A3+z4*P1;

[0069] Expanding the aforementioned formula (1), we can obtain: P1=x1*A1+x2*A2+x3*A3+x4*A4+x5*A5+x6*A6;

[0070] Combining the expanded expressions, we get: L1=(z4x1+z1)*A1+(z4x2+z2)*A2+(z4x3+z3)*A3+z4x4*A4+z4x5*A5+z4x6*A6……………(5);

[0071] According to formula (5), the first and second coefficients of the coding line corresponding to the local check block are obtained by performing the following logical calculations:

[0072] Each of the first coefficients x1 to x6 is multiplied by the first sub-coefficient z4, which is included in the second coefficient and is preset for the global check block, to obtain the third coefficients z4x1 to z4x6 of the plurality of data blocks.

[0073] Then, the third coefficients z4x1 to z4x3 of the data blocks that are within the local coding group are added to the second sub-coefficients z1 to z3 of the second coefficient, which are preset for the data blocks within the local coding group, and the third coefficients (z4x1+z1), (z4x2+z2), and (z4x3+z3) of the data blocks within the local coding group are updated.

[0074] Based on the current third coefficients (z4x1+z1), (z4x2+z2), (z4x3+z3), z4x4, z4x5, and z4x6, the second encoding row is generated.

[0075] Based on the above principles, please refer to Figure 3 , Figure 3 This is a flowchart illustrating a method for generating a second encoded line according to an embodiment of this application. Figure 3 As shown, the method may include S302-S306. In this method, the second coefficient includes a first sub-coefficient preset for the global check block and a second sub-coefficient preset for the data blocks within the local coding group.

[0076] S302, multiply each of the first coefficients by the first sub-coefficient to obtain the third coefficients of the plurality of data blocks.

[0077] This example illustrates the process by generating the second encoding line corresponding to the local parity block L2.

[0078] In this step, the first coefficients y1 to y6 can be multiplied by the first sub-coefficient r4 to obtain the third coefficients r4y1 to r4y6.

[0079] S304, add the third coefficient of the data block within the local coding group to the second sub-coefficient, and update the third coefficient of the data block within the local coding group.

[0080] In this step, the third coefficients r4y3 to r4y6 of the data blocks within the local coding group can be added to the second sub-coefficients r1 to r3 to update the third coefficients (r4y4+r1), (r4y5+z2), and (r4y6+z3) of the data blocks within the local coding group.

[0081] S306, Based on the current third coefficient, generate the second encoding line.

[0082] In this step, the third coefficients r4y1, r4y2, r4y3, (r4y4+r1), (r4y5+z2), and (r4y6+z3) can be used as elements in the second encoding line corresponding to the local check block L2 to generate the second encoding line.

[0083] Multiplying the second encoded row of L2 by the column vector consisting of the six data blocks yields:

[0084] L2=r4y1*A1+r4y2*A2+r4y3*A3+(r4y4+r1)*A4+(r4y5+r2)*A5+(r4y6+r3)*A6……………(6); This formula (6) is exactly the same as the formula obtained by combining formula (2) and formula (4), which means that the correct second coding line can be generated by S302-S306.

[0085] S206, Generate an encoding matrix based on the first encoding row and the second encoding row.

[0086] In this step, the first encoding row and the second encoding row can be used as the encoding rows of the encoding matrix to obtain the encoding matrix.

[0087] It should be noted that the encoding matrix satisfies the preset condition that if (m+1) rows are randomly selected and (m+1) columns of the elements in these (m+1) rows are selected, the resulting square matrix will definitely be a full-rank matrix; where m is the number of global check blocks.

[0088] Continuing with the previous example, the encoding matrix is:

[0089]

[0090] The encoding matrix used to generate the global parity block and the local parity block can be obtained through S202-S206.

[0091] After obtaining the encoding matrix, the encoding matrix can be acquired; then, based on the dot product of the first encoding row in the encoding matrix and the multiple data blocks, the global check block is generated; then, based on the dot product of each second encoding row in the encoding matrix and the multiple data blocks, a local check block corresponding to each local encoding group is generated.

[0092] Taking the previous example as an example, the formula can be expressed as:

[0093]

[0094] According to the aforementioned formula (7), the global check block and the local check block corresponding to each of the local coding groups can be generated.

[0095] S106, perform data repair based on the local verification block and the global verification block.

[0096] In some embodiments, the data restoration includes at least one of the following:

[0097] In the case where the global verification block needs to be repaired, the global verification block that needs to be repaired is generated based on the data blocks in the local verification block generated by the global verification block that needs to be repaired and the corresponding local verification block, so as to complete the data repair.

[0098] In the case where any data block among the plurality of data blocks needs to be repaired, the data block to be repaired is generated based on other intact data blocks in the local coding group to which the data block to be repaired belongs, the corresponding local check block, and the global check block, so as to complete the data repair.

[0099] In cases where the local verification block needs to be repaired, the local verification block to be repaired is generated based on the data block within the local verification block to be repaired and the global verification block, so as to complete the data repair.

[0100] Taking the previous example as an example, in the case where the global parity block P1 is lost or corrupted, data blocks A1 to A3 within local coding group 1, as well as local parity block L1, can be obtained. Since L1 is generated based on P1 and A1 to A3, P1 can be recovered using L1 and A1 to A3, thus completing data repair.

[0101] In the event that data block A1 is lost or corrupted, other data blocks A2 and A3 within group 1, local check block L1, and global check block P1 can be retrieved. Since L1 is generated based on P1 and A1 through A3, A1 can be recovered using L1, A2, A3, and P1, thus completing data repair.

[0102] In the event that the local parity block L2 is lost or corrupted, data blocks A4 to A6 within local coding group 2, as well as the global parity block P2, can be obtained. Since L2 is generated based on P2 and A4 to A6, L2 can be recovered using P2 and A4 to A6, thus completing data repair.

[0103] In the schemes described in S102-S106, by having the global check block participate in the generation of the local check copy, compared with LRC-related technologies, the local check block can contain information of the global check block. Thus, during the data repair process, if it is necessary to repair the global check block, the repair can be performed based on the local check block and the data blocks within the local coding group, without having to read all the data blocks. This reduces the repair traffic required to repair the global check block, thereby reducing the repair cost.

[0104] The following explanation will be based on the scenario of storing data.

[0105] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating a data storage scenario as shown in an embodiment of this application.

[0106] like Figure 4As shown, the data to be stored is divided into six data blocks, A1 to A66. These six data blocks form a global encoding group. The global encoding group corresponds to global parity blocks P1 and P2. These six data blocks are further divided into local encoding group 1 and local encoding group 2. Local encoding group 1 corresponds to local parity block L1, and local encoding group 2 corresponds to local parity block L2. L1 is generated based on P1 and A1 to A3, and L2 is generated based on P2 and A4 to A6.

[0107] Please see Figure 5 , Figure 5 This is a schematic flowchart illustrating a method for storing data according to an embodiment of this application. Figure 5 As shown, the method may include steps S501-S503. Unless otherwise specified, this application does not limit the order in which these steps are performed.

[0108] S501, obtain the preset encoding matrix and the data to be stored.

[0109] The encoding matrix includes encoding lines corresponding to global parity blocks P1 and P2, and local parity blocks L1 and L2, respectively.

[0110] The generation method of the encoding matrix can refer to S202-S206 and S302-S306, which will not be described in detail here. The encoding matrix is ​​as follows:

[0111]

[0112] The data to be stored is as follows Figure 4 As shown, it includes 6 data blocks.

[0113] S502, based on the preset check block generation formula, a global check block and a local check block are generated using the data to be stored.

[0114] In this step, a one-dimensional column vector can be obtained based on the six data blocks included in the data to be stored. Then, according to the aforementioned formula (7), global check blocks P1 and P2, as well as local check blocks L1 and L2, can be obtained.

[0115] S503, store the generated global check block and local check block, as well as the data to be stored.

[0116] In this step, the generated global verification block, local verification block, and the data to be stored can be stored in a database system. This application does not limit the type of database system; for example, the database system can be a distributed database system.

[0117] By using S501-S503, the global check block can participate in the generation of the local check copy, which, compared with LRC-related technologies, allows the local check block to contain information from the global check block.

[0118] After storage is complete, in the event that the global parity block P1 is lost or corrupted, data blocks A1 to A3 within local coding group 1, as well as local parity block L1, can be retrieved. Since L1 is generated based on P1 and A1 to A3, P1 can be recovered using L1 and A1 to A3, thus completing data repair.

[0119] As can be seen, during the data repair process, if it is necessary to repair the global check block, it can be repaired based on the local check block and the data blocks within the local coding group. There is no need to read all the data blocks, which reduces the repair traffic required to repair the global check block and thus reduces the repair cost.

[0120] Corresponding to any of the embodiments described above, this application also proposes a data repair apparatus.

[0121] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating the structure of a data repair device according to an embodiment of this application. Figure 6 As shown, the data repair device 600 may include:

[0122] The acquisition module 610 acquires multiple data blocks and multiple local coding groups obtained by grouping the multiple data blocks; each local coding group includes at least one of the data blocks.

[0123] The first generation module 620 generates a global verification block based on the plurality of data blocks, and generates a local verification block corresponding to each local coding group based on the global verification block and the data blocks contained in each local coding group.

[0124] The repair module 630 performs data repair based on the local verification block and the global verification block.

[0125] In some embodiments, the first generation module 620 further includes:

[0126] Obtain a first coefficient preset for each of the data blocks;

[0127] A global verification block is generated based on the first coefficient and the plurality of data blocks.

[0128] In some embodiments, the first generation module 620 further includes:

[0129] Obtain a second coefficient corresponding to each of the local coding groups, wherein the second coefficient indicates the preset coefficient information of the data block and the global check block within the local coding group;

[0130] Based on the second coefficient, the data blocks within the local coding group, and the global check block, a local check block is generated corresponding to each local coding group.

[0131] In some embodiments, the device 600 further includes:

[0132] Before generating a global verification block based on the first coefficient and the plurality of data blocks, and before generating a local verification block corresponding to each of the local coding groups based on the second coefficient, the data blocks within the local coding groups, and the global verification block, the second generation module generates a first coding row in the coding matrix corresponding to the global verification block based on the first coefficient; the first coding row is used to perform a dot product with the plurality of data blocks to obtain the global verification block.

[0133] Based on the first coefficient and the second coefficient corresponding to each of the local coding groups, a second coding line corresponding to each of the local coding groups is generated, so that the second coding line incorporates the first coefficient required to generate the global check block; the second coding line is used to perform a dot product with the plurality of data blocks to obtain the local check block.

[0134] An encoding matrix is ​​generated based on the first encoding row and the second encoding row.

[0135] In some embodiments, the second coefficient includes a first sub-coefficient preset for the global check block and a second sub-coefficient preset for the data blocks within the local coding group;

[0136] The second generation module further specifies the following for each of the local coding groups:

[0137] Multiply each of the first coefficients by the first sub-coefficient to obtain the third coefficients of the plurality of data blocks;

[0138] The third coefficient of the data block within the local coding group is added to the second sub-coefficient to update the third coefficient of the data block within the local coding group;

[0139] The second encoding line is generated based on the current third coefficient.

[0140] In some embodiments, the first generation module 620 further includes:

[0141] Obtain the encoding matrix;

[0142] The global check block is generated based on the dot product of the first encoded row in the encoding matrix and the multiple data blocks.

[0143] Based on the dot product of each second coding row in the coding matrix and the plurality of data blocks, a local check block corresponding to each local coding group is generated.

[0144] In some embodiments, the repair module 630 is further configured to perform at least one of the following:

[0145] In the case where the global verification block needs to be repaired, the global verification block that needs to be repaired is generated based on the data blocks in the local verification block generated by the global verification block that needs to be repaired and the corresponding local verification block, so as to complete the data repair.

[0146] In the case where any data block among the plurality of data blocks needs to be repaired, the data block to be repaired is generated based on other intact data blocks and corresponding local check blocks within the local coding group to which the data block to be repaired belongs, so as to complete the data repair.

[0147] In cases where the local verification block needs to be repaired, the local verification block to be repaired is generated based on the data block within the local verification block to be repaired and the global verification block, so as to complete the data repair.

[0148] In the aforementioned scheme, by allowing the global check block to participate in the generation of the local check copy, compared with LRC-related technologies, the local check block can contain information of the global check block. Thus, during the data repair process, if it is necessary to repair the global check block, the repair can be performed based on the local check block and the data blocks within the local coding group, without having to read all the data blocks. This reduces the repair traffic required to repair the global check block, thereby reducing the repair cost.

[0149] The embodiments of the data repair apparatus shown in this application can be applied to electronic devices. Accordingly, this application discloses an electronic device that may include: a processor; and a memory for storing processor-executable instructions. The processor is configured to invoke the executable instructions stored in the memory to implement the data repair method shown in any of the foregoing embodiments.

[0150] Please see Figure 7 , Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application.

[0151] like Figure 7 As shown, the electronic device may include a processor for executing instructions, a network interface for network connectivity, memory for storing operational data for the processor, and non-volatile memory for storing instructions corresponding to the data repair device.

[0152] The data repair device can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of its electronic device reading the corresponding computer program instructions from non-volatile memory into memory and executing them. From a hardware perspective, besides... Figure 7 In addition to the processor, memory, network interface, and non-volatile memory shown, the electronic device in which the device is located in the embodiment may also include other hardware depending on the actual function of the electronic device, which will not be described in detail here.

[0153] It is understandable that, in order to improve processing speed, the corresponding instructions of the data repair device can also be directly stored in memory, which is not limited here.

[0154] This application proposes a computer-readable storage medium storing a computer program that can be used to cause a processor to execute the data repair method shown in any of the foregoing embodiments.

[0155] Those skilled in the art will understand that one or more embodiments of this application can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (which may include, but are not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] In this application, “and / or” means having at least one of two options. For example, “A and / or B” can include three options: A, B, and “A and B”.

[0157] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the data processing device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0158] The specific embodiments of this application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0159] The embodiments of the subject matter and functional operation described in this application can be implemented in: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware that may include the structures disclosed in this application and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this application can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0160] The processing and logic flow described in this application can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGA (Field-Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit)—and the device can also be implemented as dedicated logic circuitry.

[0161] A computer suitable for executing computer programs may include, for example, a general-purpose and / or special-purpose microprocessor, or any other type of processing unit. Typically, the processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer may include a processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer may also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer may be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0162] Computer-readable media suitable for storing computer program instructions and data can include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROMs and DVD-ROMs. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0163] While this application contains numerous specific implementation details, these should not be construed as limiting the scope of any disclosure or the scope of the claims, but rather are primarily used to describe the features of specific embodiments of a particular disclosure. Certain features described in the multiple embodiments of this application may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0164] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the described embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0165] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0166] The above are merely preferred embodiments of one or more embodiments of this application and are not intended to limit the scope of one or more embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this application should be included within the scope of protection of one or more embodiments of this application.

Claims

1. A data repair method characterized by, The method comprises the following steps: obtaining a plurality of data blocks and a plurality of local encoding groups obtained by grouping the plurality of data blocks; the local encoding group comprises at least one data block; based on the plurality of data blocks, generating a global check block, and generating a local check block corresponding to each local encoding group according to the global check block and the data blocks contained in each local encoding group; based on the local check block and the global check block, data repair is performed; before generating a global check block based on the plurality of data blocks, and generating a local check block corresponding to each local encoding group according to the global check block and the data blocks contained in each local encoding group, the method further comprises the following steps: based on the first coefficient preset for each data block and the second coefficient corresponding to each local encoding group, a second encoding row corresponding to each local encoding group is generated, wherein the second coefficient comprises a first sub-coefficient preset for the global check block and a second sub-coefficient preset for the data blocks in the local encoding group; for each local encoding group: multiply each first coefficient by the first sub-coefficient to obtain a third coefficient of the plurality of data blocks; add the third coefficient of the data blocks in the local encoding group in the plurality of data blocks to the second sub-coefficient to update the third coefficient of the data blocks in the local encoding group; based on the current third coefficient, the second encoding row is generated, and the second encoding row is used to generate the local check block corresponding to each local encoding group.

2. The method of claim 1, wherein, The method comprises the following steps: obtaining a plurality of data blocks and a plurality of local encoding groups obtained by grouping the plurality of data blocks; the local encoding group comprises at least one data block; 3. The method of claim 2, wherein, based on the plurality of data blocks, generating a global check block, and generating a local check block corresponding to each local encoding group according to the global check block and the data blocks contained in each local encoding group; based on the local check block and the global check block, data repair is performed; before generating a global check block based on the plurality of data blocks, and generating a local check block corresponding to each local encoding group according to the global check block and the data blocks contained in each local encoding group, the method further comprises the following steps:

4. The method of claim 3, wherein, based on the first coefficient preset for each data block and the second coefficient corresponding to each local encoding group, a second encoding row corresponding to each local encoding group is generated, wherein the second coefficient comprises a first sub-coefficient preset for the global check block and a second sub-coefficient preset for the data blocks in the local encoding group; for each local encoding group: multiply each first coefficient by the first sub-coefficient to obtain a third coefficient of the plurality of data blocks; add the third coefficient of the data blocks in the local encoding group in the plurality of data blocks to the second sub-coefficient to update the third coefficient of the data blocks in the local encoding group; based on the current third coefficient, the second encoding row is generated, and the second encoding row is used to generate the local check block corresponding to each local encoding group. The method comprises the following steps: obtaining a plurality of data blocks and a plurality of local encoding groups obtained by grouping the plurality of data blocks; the local encoding group comprises at least one data block; based on the plurality of data blocks, generating a global check block, and generating a local check block corresponding to each local encoding group according to the global check block and the data blocks contained in each local encoding group; based on the local check block and the global check block, data repair is performed; before generating a global check block based on the plurality of data blocks, and generating a local check block corresponding to each local encoding group according to the global check block and the data blocks contained in each local encoding group, the method further comprises the following steps: based on the first coefficient preset for each data block and the second coefficient corresponding to each local encoding group, a second encoding row corresponding to each local encoding group is generated, wherein the second coefficient comprises a first sub-coefficient preset for the global check block and a second sub-coefficient preset for the data blocks in the local encoding group; for each local encoding group: multiply each first coefficient by the first sub-coefficient to obtain a third coefficient of the plurality of data blocks; add the third coefficient of the data blocks in the local encoding group in the plurality of data blocks to the second sub-coefficient to update the third coefficient of the data blocks in the local encoding group; based on the current third coefficient, the second encoding row is generated, and the second encoding row is used to generate the local check block corresponding to each local encoding group. generate a coding matrix based on the first coding row and the second coding row.

5. The method of claim 4, wherein, The generating the global check block based on the first coefficients and the data blocks, and generating the local check block corresponding to each of the local coding groups based on the second coefficients and the data blocks in the local coding groups and the global check block comprises: obtaining the coding matrix; generating the global check block based on a dot product operation result of the first coding row in the coding matrix and the data blocks; generating the local check block corresponding to each of the local coding groups based on a dot product operation result of each of the second coding rows in the coding matrix and the data blocks.

6. The method of claim 1, wherein, The data repairing based on the local check block and the global check block comprises at least one of: in a case that the global check block needs to be repaired, generating the global check block that needs to be repaired based on the data blocks in the local check block generated by the global check block that needs to be repaired and the corresponding local check block, so as to complete the data repairing; in a case that any data block in the data blocks needs to be repaired, generating the data block that needs to be repaired based on the other intact data blocks in the local coding group to which the data block that needs to be repaired belongs and the corresponding local check block, so as to complete the data repairing; in a case that the local check block needs to be repaired, generating the local check block that needs to be repaired based on the data blocks in the local check block that needs to be repaired and the global check block, so as to complete the data repairing.

7. A data recovery apparatus, characterized by comprising: comprise: an obtaining module, which obtains a plurality of data blocks and a plurality of local coding groups obtained by grouping the data blocks; the local coding group comprises at least one data block; a generating module, which generates a global check block based on the data blocks, and generates a local check block corresponding to each of the local coding groups based on the global check block and the data blocks included in each of the local coding groups; a repairing module, which performs data repairing based on the local check block and the global check block; a second generating module, which generates a second coding row corresponding to each of the local coding groups based on a first coefficient preset for each of the data blocks and a second coefficient corresponding to each of the local coding groups, wherein the second coefficient comprises a first sub-coefficient preset for the global check block and a second sub-coefficient preset for the data blocks in the local coding group; the second generating module further comprises, for each of the local coding groups: multiplying the first coefficient and the first sub-coefficient to obtain a third coefficient of the data blocks; adding the third coefficient of the data blocks in the local coding group and the second sub-coefficient to update the third coefficient of the data blocks in the local coding group; generating the second coding row based on the current third coefficient.

8. An electronic device, comprising: comprise: a processor; a memory for storing processor-executable instructions; wherein the processor implements the data repairing method according to any one of claims 1-6 by running the executable instructions. comprise: a processor; a memory for storing processor-executable instructions; wherein the processor implements the data repairing method according to any one of claims 1-6 by running the executable instructions.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for causing the processor to execute the data repair method according to any one of claims 1-6.

Images