Data block encoding and data block recovery

Abstract

Provided in the present application are a data block encoding method, a data block recovery method, an apparatus and an electronic device. The data block encoding method comprises: partitioning a plurality of original data blocks to obtain a plurality of groups, the plurality of groups comprising at least one cross block, and the cross block being the same original data block in the plurality of groups; on the basis of the number of the original data blocks and the number of the groups, constructing a Cauchy matrix; and generating a global parity block of each group on the basis of the Cauchy matrix and the plurality of original data blocks, generating a local parity block of each group on the basis of the Cauchy matrix, the plurality of original data blocks, a preset cross block coefficient and the cross block, and adding the global parity block and the local parity block of each group into the group, the global parity blocks of the plurality of groups being added to form a new global parity block.

Description

Technical Field of Data Block Encoding and Data Block Recovery

[0001] This application relates to the field of data storage, and more specifically, to data block encoding and data block recovery. Background Art

[0002] Erasure Code is a coding fault tolerance technology. Its basic principle is to fragment the stored data, and generate k + m pieces of data from k pieces of original data through a certain parity calculation method, and the original data can be restored through any k pieces of the k + m pieces of data. In this way, even if some data is lost, the system can still recover the original data. Due to the characteristics of Erasure Code that it occupies less overall storage space of the cluster while ensuring reliability, it is widely used in almost all distributed storage systems. However, compared with the traditional triple replication, Erasure Code has the problem of read amplification during data reconstruction.

[0003] Local Repair Code (LRC) is a local parity erasure code method. Its core idea is: divide the data blocks into multiple groups, and divide the parity blocks into global parity and local parity. When the number of lost data blocks does not exceed the number of local parity blocks within the group, data recovery can be performed through other readable data blocks within the group where the data block is located, which can reduce the problem of read amplification.

[0004] Currently, LRC has been increasingly widely used in various storage systems, but the problem existing in various LRC schemes is that the lost data blocks can only be locally recovered through one group, and the local recovery mode is single, which is likely to cause too long recovery delay. Summary of the Invention

[0005] This application provides a data block encoding method, a data block recovery method, a device and an electronic device to achieve multiple local recovery modes and avoid too long recovery delay.

[0006] In a first aspect, the present application provides a data block encoding method, including: dividing a plurality of original data blocks to obtain a plurality of groups, where at least one cross block is included in the plurality of groups, and the cross block is the same original data block in the plurality of groups; constructing a Cauchy matrix based on the number of the original data blocks and the number of the groups; generating a global check block for each group based on the Cauchy matrix and the plurality of original data blocks, generating a local check block for each group based on the Cauchy matrix, the plurality of original data blocks, a preset cross block coefficient, and the cross block, and adding the global check block and the local check block of each group to the each group, where the sum of the global check blocks of the plurality of groups forms a new global check block. The sum of the global check blocks forms a new global check block.

[0007] In one implementation, the constructing a Cauchy matrix based on the number of the original data blocks and the number of the groups includes: using the number of the groups plus 1 as the number of rows and the number of the original data blocks as the number of columns to construct the Cauchy matrix.

[0008] In one implementation, the generating a global check block for each group based on the Cauchy matrix and the plurality of original data blocks includes: for the i-th group, multiplying the i-th row of the Cauchy matrix by the vector formed by the plurality of original data blocks to obtain the global check block of the i-th group, where the value of i ranges from 1 to the number of the groups.

[0009] In one implementation, generating local check blocks for each group based on the Cauchy matrix, the multiple original data blocks, preset cross-block coefficients, and the cross blocks includes: summing the data of each row of the Cauchy matrix by column to obtain the Cauchy matrix sum values corresponding to each column, and the Cauchy matrix sum values corresponding to each column correspond to the original data blocks; for the i-th group, determining the local check block of the i-th group in the following manner, where the value of i ranges from 1 to the number of groups: multiplying the i-th row of the Cauchy matrix by the vector formed by the multiple original data blocks, adding the sum of the Cauchy matrix sum values corresponding to the i-th group multiplied by the sum of the target original data blocks corresponding to the i-th group, and adding the sum of the cross blocks multiplied by the i-th cross-block coefficient to obtain the local check block of the i-th group; where the target original data blocks corresponding to the i-th group are part or all of the original data blocks included in the i-th group, the target original data blocks corresponding to the multiple groups include all the original data blocks and the target original data blocks corresponding to each group do not repeat; the Cauchy matrix sum values corresponding to the i-th group refer to the Cauchy matrix sum values corresponding to the target original data blocks corresponding to the i-th group; and the sum of the cross-block coefficients is zero.

[0010] In a second aspect, the present application provides a data block recovery method, including: determining a lost target data block; where the target data block is a data block in a group obtained by encoding multiple original data blocks using the method described in the first aspect; if the target data block is any cross block, then using the data blocks in any group to which the cross block belongs to recover the target data block.

[0011] In one implementation, it further includes: when using the data blocks in any group to which the cross block belongs to recover the target data block, if the recovery is not completed within a preset time, then reusing the data blocks in other groups to which the cross block belongs to recover the target data block.

[0012] In one implementation, it further includes: if the target data block is any data block other than the cross block, then recovering the target data block based on the data blocks in the group to which the target data block belongs.

[0013] In one implementation, it further includes: if the target data block is any n data blocks, where the value of n ranges from 2 to the number of groups plus 1, then recovering the target data block based on the data blocks in each group.

[0014] In a third aspect, the present application provides a data block encoding device, including: a grouping module, configured to divide a plurality of original data blocks to obtain a plurality of groups, where at least one cross block is included in the plurality of groups, and the cross block is the same original data block in the plurality of groups; a construction module, configured to construct a Cauchy matrix based on the number of the original data blocks and the number of the groups; an encoding module, configured to generate a global check block for each group based on the Cauchy matrix and the plurality of original data blocks, generate a local check block for each group based on the Cauchy matrix, the plurality of original data blocks, a preset cross block coefficient, and the cross block, and add the global check block and the local check block of each group to each group, where the sum of the global check blocks of the plurality of groups constitutes a new global check block.

[0015] In one implementation, the construction module is configured to: use the number of the groups plus 1 as the number of rows and the number of the original data blocks as the number of columns to construct the Cauchy matrix.

[0016] In one implementation, the encoding module is configured to: for the i-th group, multiply the i-th row of the Cauchy matrix by the vector formed by the plurality of original data blocks to obtain the global check block of the i-th group, where i ranges from 1 to the number of the groups.

[0017] In one implementation, the encoding module is configured to: sum the data of each row of the Cauchy matrix by column to obtain the Cauchy matrix sum value corresponding to each column, and the Cauchy matrix sum value corresponding to each column corresponds to each original data block; for the i-th group, determine the local check block of the i-th group in the following manner, where i ranges from 1 to the number of the groups: multiply the i-th row of the Cauchy matrix by the vector formed by the plurality of original data blocks, add the sum of the Cauchy matrix sum values corresponding to the i-th group multiplied by the sum of the target original data blocks corresponding to the i-th group, and then add the sum of the i-th cross block coefficient multiplied by each cross block to obtain the local check block of the i-th group; where the target original data blocks corresponding to the i-th group are part or all of the original data blocks included in the i-th group, the target original data blocks corresponding to the plurality of groups include all the original data blocks and the target original data blocks corresponding to each group do not repeat; the Cauchy matrix sum values corresponding to the i-th group refer to the Cauchy matrix sum values corresponding to the target original data blocks corresponding to the i-th group; and the sum of each cross block coefficient is zero.

[0018] Fourth aspect, the present application provides a data block recovery device, including: a determination module, configured to determine a lost target data block; wherein, the target data block is a data block in a group obtained by encoding a plurality of original data blocks by using the method described in the first aspect; a recovery module, configured to, if the target data block is any cross block, recover the target data block by using the data blocks in any group to which the cross block belongs.

[0019] In one implementation, the recovery module is configured to: when recovering the target data block by using the data blocks in any group to which the cross block belongs, if the recovery is not completed within a preset time, then recover the target data block again by using the data blocks in other groups to which the cross block belongs.

[0020] In one implementation, the recovery module is configured to: if the target data block is any data block other than a cross block, then recover the target data block based on the data blocks in the group to which the target data block belongs.

[0021] In one implementation, the recovery module is configured to: if the target data block is any n data blocks, where n ranges from 2 to the number of groups plus 1, then recover the target data block based on the data blocks in each group.

[0022] Fifth aspect, the present application provides an electronic device, including: a memory and a processor; the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory, and when the computer program runs, the processor executes the method described in the first aspect or the second aspect.

[0023] Sixth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the processor is caused to execute the method described in the first aspect or the second aspect.

[0024] Seventh aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0025] ​In the data block encoding method, data block recovery method, device and electronic device provided in this application, the original data blocks are grouped, and some of the original data blocks are divided into cross blocks, that is, the same data blocks in each group. Then, a Cauchy matrix is constructed based on the number of groups and the number of original data blocks, and local check blocks and global check blocks for each group are further generated. In the case of the loss of cross blocks, recovery can be performed through the data in any group, providing more flexible options for data recovery and avoiding problems such as high recovery latency caused by a single local recovery mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0027] FIG. 1 is a schematic diagram of data blocks in an LRC mode;

[0028] FIG. 2 is a schematic diagram of data blocks in another LRC mode;

[0029] FIG. 3 is a schematic diagram of data blocks in an LRC mode provided by an embodiment of this application;

[0030] FIG. 4 is a schematic flowchart of a data block encoding method provided by an embodiment of this application;

[0031] FIG. 5 is a first example of data block grouping provided by an embodiment of this application;

[0032] FIG. 6 is a second example of data block grouping provided by an embodiment of this application;

[0033] FIG. 7 is a schematic flowchart of a data block recovery method provided by an embodiment of this application;

[0034] FIG. 8 is a schematic structural diagram of a data block encoding device provided by an embodiment of this application;

[0035] FIG. 9 is a schematic structural diagram of a data block recovery device provided by an embodiment of this application;

[0036] FIG. 10 is a schematic block diagram of an electronic device provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.

[0038] Figure 1 is a schematic diagram of data blocks in an LRC mode. As shown in Figure 1, each group includes multiple original data blocks. For example, Group 1 includes the original Group g includes original data blocks D&i to D&r. Each group has a local parity block. For example, the local parity block Lj in Group 1 and the local parity block Lg in Group g. The global parity blocks are stored separately. For example, global parity blocks G] to Gg.

[0039] Figure 2 is a schematic diagram of data blocks in another LRC mode. As shown in Figure 2, each group includes multiple original data blocks. For example, Group 1 includes original data blocks %,1 to %,r, and Group g includes original data blocks. Hong to D&r. Each group has a local parity block and a global parity block. For example, the local parity block d and the global parity block G] in Group 1, and the local parity block Lg and the global parity block Gg in Group g.

[0040] Regardless of which of the above LRC modes, each data block has only one local recovery mode. For example, if %,1 in Figure 1 or Figure 2 is lost, only other data blocks in Group 1 can be used to perform local recovery, and the local recovery mode is single. The problem that may be caused by this single local recovery mode is that if there are slow disks among other data blocks in Group 1, then even if local recovery is used, the data recovery delay will be relatively long.

[0041] In view of this, an embodiment of this application proposes a data block encoding method, providing a new LRC mode, dividing some original data blocks into each group, that is, each group contains the same original data blocks. For example, the 3 original data blocks located in the center of the figure shown in Figure 3 are included in Group Group 2 to Group 1. For example For example, the data blocks corresponding to the disks prone to failure are divided into cross blocks, that is, the same data blocks in each Group, and then the corresponding local parity blocks and global parity blocks are generated. In this way, if a cross block is lost, there are g local recovery modes, where g is the number of Groups, providing more flexible choices for data recovery and avoiding problems such as high recovery latency caused by a single local recovery mode.

[0042] FIG. 4 is a schematic flowchart of a data block encoding method provided by an embodiment of the present application. As shown in FIG. 4, the method includes steps S401 to S403.

[0043] S401. Divide a plurality of original data blocks to obtain a plurality of groups, where at least one cross block is included in the plurality of groups, and the cross block is the same original data block in the plurality of groups.

[0044] In a distributed system, data is stored in the form of data blocks. All the data to be stored in a file to be stored or a storage request of a user will be decomposed into a plurality of original data blocks to be stored. The decomposed plurality of original data blocks can be stored on different nodes in the distributed system or on different disks of the same node device.

[0045] In the embodiment of the present application, in order to implement multiple local recovery modes, when grouping the original data blocks, some original data blocks are divided into each group so that at least one same original data block, that is, a cross block, is included in each group. The cross block can be an original data block prone to failure, that is, prone to loss.

[0046] Optionally, when grouping, the plurality of original data blocks can be grouped without repetition first. At this time, at least one original data block is included in each group, but the original data blocks in different groups are not repeated. Then, the original data blocks serving as cross blocks are determined from the original data blocks. In addition to being divided into its original group, the cross block is also divided into other groups other than its original group. For example, the number of original data blocks is 5, respectively For the first group, D4 and D5 are divided into the second group. Then, it is determined that % of these 5 original data blocks are cross blocks, and then the cross block D1 is divided into the second group. In this way, finally two groups are obtained, one group

[0047] S402. Construct a Cauchy matrix based on the number of original data blocks and the number of groups.

[0048] For example, taking the number of original data blocks as k and the number of groups as r, a Cauchy matrix is constructed with the number of groups plus 1 as the number of rows and the number of original data blocks as the number of columns, that is, a (r + 1) * k Cauchy matrix is constructed.

[0049] S403. Generate global check blocks for each group based on the Cauchy matrix and multiple original data blocks, generate local check blocks for each group based on the Cauchy matrix, multiple original data blocks, a preset cross-block coefficient, and cross blocks, and add the global check blocks and local check blocks of each group to each group, where the sum of the global check blocks of multiple groups constitutes a new global check block.

[0050] In the implementation of this application, corresponding global check blocks and local check blocks are generated for each group, and the global check blocks and local check blocks are added to their corresponding groups. Thus, if any database in a group is lost, it can be restored through other databases in the group. For cross blocks, since cross blocks belong to multiple groups, cross blocks can be restored through the data in any group. Moreover, the sum of the global check blocks of multiple groups in the embodiment of this application constitutes a new global check block, which is equivalent to having r + 1 global check blocks. Therefore, data recovery can be achieved when the number of lost data blocks in all groups is less than or equal to r + 1.

[0051] The global check blocks and local check blocks are described below.

[0052] For the i-th group, multiply the i-th row of the Cauchy matrix by the vector formed by multiple original data blocks to obtain the global check block of the i-th group, where the value of i ranges from 1 to the number of groups r. That is, the operation is performed according to the following formula:

[0053] Among them, represents the global check block of the i-th group, Dj represents the j-th original data block, ai,j represents the value of the i-th row and j-th column of the Cauchy matrix, and k is the number of original data blocks.

[0054] Sum the data of each row of the Cauchy matrix by column to obtain the Cauchy matrix sum value corresponding to each column, and the Cauchy matrix sum value corresponding to each column corresponds to each original data block. That is, the operation is performed according to the following formula:

[0055] Among them, ai,j represents the value of the i-th row and j-th column of the Cauchy matrix, k is the number of original data blocks, r is the number of groups, dj represents the Cauchy matrix sum value corresponding to the j-th column of the Cauchy matrix, and dj corresponds to Dj.

[0056] For the i-th group, the local check block of the i-th group is determined as follows, where the value of i ranges from 1 to the number of groups r.

[0057] The i-th row of the Cauchy matrix is multiplied by a vector composed of multiple original data blocks, and the sum of the Cauchy matrices corresponding to the i-th group is added by the sum of the target original data blocks corresponding to the i-th group, and the i-th cross block coefficient is added by the sum of the cross blocks to obtain the local check block of the i-th group; wherein, the target original data block corresponding to the i-th group is part or all of the original data blocks contained in the i-th group, the target original data blocks corresponding to the multiple groups include all the original data blocks and the target original data blocks corresponding to the groups are not repeated. For example, according to the grouping method of the aforementioned embodiment, the multiple original data blocks are first grouped without repetition, and then the cross blocks are determined, and the cross blocks are divided into each group. In this case, the target original data block corresponding to the i-th group is the original data block in each group after the non-repetitive grouping, that is, the original data block unique to each group before the cross blocks are added to each group; the sum of the Cauchy matrices corresponding to the i-th group refers to the Cauchy matrix corresponding to each target original data block corresponding to the i-th group. The sum of the cross-block coefficients is zero. That is, the calculation is performed according to the following formula:

[0058] Wherein, Li represents the local check block of the i-th group, Dj represents the j-th original data block, ai,j represents the value of the i-th row and j-th column of the Cauchy matrix, and k is the number of original data blocks.

[0059] Mi represents the number of target original data blocks corresponding to the i-th group, that is, the target original data blocks contained in each group

[0060] The number of cross blocks is C, C > 1, / represents the number of the cross block, 1 < Vj < k, Xi represents the preset cross block coefficient, Xi satisfies the operation domain, and the sum of r Xi is 0, that is, £旗晃*顼. For example, in GF (2 A 8), r can take the following values: if r is an even number, Xi is 1; if r is an odd number, The remaining Xj are 1.

[0061] The following is an explanation with reference to specific examples.

[0062] Taking the number of original data blocks as 5 and the number of groups as 2 as an example, as shown in Figure 5, one group includes the original

[0064] Sum the rows of the Cauchy matrix by columns to obtain the Cauchy matrix sum value d:

[0065] Calculate the global parity block and local parity blocks according to the following formula:

[0066] where the values of i are 1 and 2 respectively.

[0067] The LRC encoding matrix corresponding to the above operation formula is as follows: The target original data blocks corresponding to 2 groups are and for illustration. If the target original data blocks of the two groups are different from the above example, for example, the situation of the original data blocks unique to each group before adding the cross blocks is different from the above example, the target original data blocks corresponding to the first group are and D3, and the target original data blocks corresponding to the second group are %, D4, and D5, then the part of the Cauchy matrix sum value multiplied by the target original data blocks in the above formula can be adjusted accordingly.

[69] In this embodiment, the data blocks included in the final two groups are respectively: Recover the other data blocks in the group where the block is located. If % is lost, it can be recovered through the other data blocks in any one of the two groups. Moreover, the sum of the two local parity blocks is equivalent to a global parity block, which can further ensure that any three database losses can be recovered.

[70] Taking the number of original data blocks as 9 and the number of groups as 3 as an example, as shown in Figure 6, one group includes the original

[72] Sum the rows of the Cauchy matrix by columns to obtain the Cauchy matrix sum value d:

[73] Calculate the global parity block and local parity blocks according to the following formula:

[74] The LRC encoding matrix corresponding to the above operation formula is as follows: When the target original data blocks corresponding to the 2 groups are D6 and D7, and the target original data block corresponding to the 3rd group is D8, D9 and D10 are taken as examples. If the target original data blocks of the three groups are different from the above examples, for example, the situation of the original data blocks unique to each group before adding the cross blocks is different from the above examples, the target original data block corresponding to the 1st group is Just make corresponding adjustments.

[0076] In this embodiment, the data blocks included in the final three groups are respectively: When a single data block is lost, it can be recovered through other data blocks in the group where the lost data block is located. If any one of D2, D4 or D7 is lost, it can be recovered through other data blocks in any of the three groups. Moreover, the sum of the three local check blocks is equivalent to a global check block, so that any four data blocks can be recovered.

[0077] It should be noted that the addition or summation described in the embodiments of the present application can be implemented by exclusive OR operation.

[0078] Based on the above data block encoding, the embodiments of the present application also provide a data block recovery method. FIG. 7 is a schematic flowchart of a data block recovery method provided by the embodiments of the present application. As shown in FIG. 7, the method includes step S701 and step S702.

[0079] S701. Determine the lost target data block.

[0080] Wherein, the target data block is the data block in the group obtained by encoding a plurality of original data blocks by the method of the foregoing embodiment.

[0081] S702. If the target data block is any cross block, then use the data block in any group to which the cross block belongs to recover the target data block.

[0082] In the case of grouping the original data blocks and generating the global check block and local check blocks by the method of the foregoing embodiment, since the cross block belongs to multiple groups, in the case of the loss of the cross block, the data block in any group can be used for recovery, that is, there are multiple local recovery modes, which can be flexibly selected in the application. For example, when a certain group has a slow disk, other groups can be selected for data recovery.

[0083] In practical applications, a timeout can be set for data recovery. Optionally, when recovering a target data block using data blocks in any one of the groups to which the cross block belongs, if the recovery is not completed within the preset time, the target data block is recovered again using data blocks in other groups to which the cross block belongs. Thus, it is possible to avoid excessive recovery delay and improve the recovery efficiency.

[0084] Optionally, if the target data block is any data block other than the cross block, the target data block is recovered based on the data blocks in the group to which the target data block belongs.

[0085] Optionally, if the target data block is any n data blocks, where n ranges from 2 to the number of groups plus 1, the target data block is recovered based on the data blocks in each group.

[0086] The target data block can be an original data block, a local parity block, or a global parity block.

[0087] It should be noted that the local parity block and the global parity block are obtained through formula operations in the foregoing embodiments. Therefore, during data recovery, the operations can be performed with reference to the foregoing formulas.

[0088] FIG. 8 is a schematic structural diagram of a data block encoding device provided by an embodiment of the present application. As shown in FIG. 8, the data block encoding device 800 includes a grouping module 801, a construction module 802, and an encoding module 803.

[0089] The grouping module 801 is configured to divide a plurality of original data blocks into a plurality of groups, where at least one cross block is included in the plurality of groups, and the cross block is the same original data block in the plurality of groups.

[0090] The construction module 802 is configured to construct a Cauchy matrix based on the number of original data blocks and the number of groups.

[0091] The encoding module 803 is configured to generate global parity blocks for each group based on the Cauchy matrix and the plurality of original data blocks, generate local parity blocks for each group based on the Cauchy matrix, the plurality of original data blocks, a preset cross block coefficient, and the cross block, and add the global parity blocks and local parity blocks of each group to each group, where the global parity blocks of the plurality of groups are added together to form a new global parity block.

[0092] In one implementation, the construction module 802 is configured to: construct a Cauchy matrix with the number of groups plus 1 as the number of rows and the number of original data blocks as the number of columns.

[0093] In one implementation, the encoding module 803 is configured to: for the i-th packet, multiply the i-th row of the Cauchy matrix by a vector formed by a plurality of original data blocks to obtain a global check block of the i-th packet, where the value of i ranges from 1 to the number of packets.

[0094] In one implementation, the encoding module 803 is configured to: sum the data of each row of the Cauchy matrix by column to obtain the Cauchy matrix sum value corresponding to each column, and the Cauchy matrix sum value corresponding to each column corresponds to each original data block; for the i-th packet, determine the local check block of the i-th packet in the following manner, where the value of i ranges from 1 to the number of packets: multiply the i-th row of the Cauchy matrix by a vector formed by a plurality of original data blocks, add the sum of the Cauchy matrix sum values corresponding to the i-th packet multiplied by the sum of the target original data blocks corresponding to the i-th packet, and then add the product of the i-th cross-block coefficient and the sum of each cross-block to obtain the local check block of the i-th packet; where the target original data block corresponding to the i-th packet is part or all of the original data blocks included in the i-th packet, the target original data blocks corresponding to a plurality of packets include all the original data blocks and the target original data blocks corresponding to each packet do not repeat; the Cauchy matrix sum values corresponding to the i-th packet refer to the Cauchy matrix sum values corresponding to the target original data blocks corresponding to the i-th packet; the sum of each cross-block coefficient is zero.

[0095] The device according to the embodiment of the present application can be used to execute the data block encoding method in the foregoing embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0096] FIG. 9 is a schematic structural diagram of a data block recovery device provided by an embodiment of the present application. As shown in FIG. 9, the data block recovery device 900 includes: a determination module 901, configured to determine a lost target data block; where the target data block is a data block in a packet obtained by encoding a plurality of original data blocks by using the method in the first aspect; a recovery module 902, configured to, if the target data block is any cross-block, recover the target data block by using the data blocks in any packet to which the cross-block belongs.

[0097] In one implementation, when the recovery module 902 uses the data blocks in any packet to which the cross-block belongs to recover the target data block, if the recovery is not completed within a preset time, the recovery module 902 reuses the data blocks in other packets to which the cross-block belongs to recover the target data block.

[0098] In one implementation, the recovery module 902 is configured to: if the target data block is any data block other than the cross block, recover the target data block based on the data blocks in the group to which the target data block belongs.

[0099] In one implementation, the recovery module 902 is configured to: if the target data block is any n data blocks, where n ranges from 2 to the number of groups plus 1, recover the target data block based on the data blocks in each group.

[0100] The device according to the embodiments of the present application can be used to execute the data block recovery method in the foregoing embodiments. The implementation principle and technical effects are similar, and will not be elaborated here.

[0101] FIG. 10 is a schematic block diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 10, the electronic device 1000 may include at least one processor 1001, which is configured to implement the method for reading multi-copy data provided by the embodiments of the present application.

[0102] Optionally, the electronic device 1000 further includes at least one memory 1002, which is configured to store program instructions and / or data. The memory 1002 is coupled to the processor 1001. The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units or modules, which may be electrical, mechanical or other forms, and is used for information interaction between devices, units or modules. The processor 1001 may cooperate with the memory 1002. The processor 1001 may execute the program instructions stored in the memory 1002. At least one of the at least one memories may be included in the processor.

[0103] Optionally, the electronic device 1000 further includes a communication interface 1003, which is configured to communicate with other devices through a transmission medium, so that the electronic device 1000 can communicate with other devices. The communication interface 1003 may be, for example, a transceiver, an interface, a bus, a circuit or a device capable of implementing a transceiver function. The processor 1001 may use the communication interface 1003 to transmit and receive data and / or information, and is configured to implement the method provided by the embodiments of the present application. For specific details, refer to the detailed description in the foregoing embodiments, and will not be elaborated here.

[0104] In the embodiments of the present application, the specific connection medium between the above-mentioned processor 1001, memory 1002, and communication interface 1003 is not limited. In the embodiments of the present application, in FIG. 10, the processor 1001, memory 1002, and communication interface 1003 are connected through a bus 1004. The bus 1004 is represented by a thick line in FIG. 10. The connection manners between other components are only for illustrative purposes and are not to be taken as a limitation. This bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, only one thick line is used in FIG. 10, but it does not mean that there is only one bus or one type of bus.

[0105] It should be understood that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiments can be completed by the integrated logic circuit in the hardware of the processor or instructions in software form. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or can be executed and completed by a combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0106] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (read - only memory, ROM), programmable read - only memory (PROM), erasable programmable read - only memory (EPROM), electrically erasable programmable read - only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random - access memory (SRAM), dynamic random - access memory (DRAM), synchronous dynamic random - access memory (SDRAM), double data rate synchronous dynamic random - access memory (DDR SDRAM), enhanced synchronous dynamic random - access memory (ESDRAM), synchlink dynamic random - access memory (SLDRAM), and direct rambus random - access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include but not be limited to these and any other suitable types of memory.

[0107] This application also provides a computer - readable storage medium that stores a computer program (which can also be referred to as code or instructions). When the computer program is run, it causes the computer to execute the method in any of the foregoing embodiments.

[0108] This application also provides a computer program product that includes a computer program, and when the computer program is executed by a processor, it implements the method in any of the foregoing embodiments.

[0109] The terms "unit", "module", etc. used in this specification can be used to represent computer - related entities, hardware, firmware, combinations of hardware and software, software, or software in execution.

[0110] Those of ordinary skill in the art can realize that the various illustrative logical blocks and steps described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application. In several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be in electrical, mechanical, or other forms.

[0111] The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, each functional unit in the various embodiments of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0113] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that incorporates one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0114] If this function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the related technology, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0115] The user information involved in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0116] As described above, it is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claimed rights.

Claims

Claims 1. A data block encoding method, comprising: Divide multiple original data blocks to obtain multiple groups, where at least one cross block is included in the multiple groups, and the cross block is the same original data block in the multiple groups; Construct a Cauchy matrix based on the number of the original data blocks and the number of the groups; Generate global check blocks for each group based on the Cauchy matrix and the multiple original data blocks, generate local check blocks for each group based on the Cauchy matrix, the multiple original data blocks, a preset cross block coefficient, and the cross blocks, and add the global check blocks and local check blocks of each group to each group, where the sum of the global check blocks of the multiple groups constitutes a new global check block.

2. The method according to claim 1, wherein, The constructing a Cauchy matrix based on the number of the original data blocks and the number of the groups includes: using the number of the groups plus 1 as the number of rows and the number of the original data blocks as the number of columns to construct the Cauchy matrix.

3. The method according to claim 2, wherein, The generating global check blocks for each group based on the Cauchy matrix and the multiple original data blocks includes: for the i-th group, multiplying the i-th row of the Cauchy matrix by the vector formed by the multiple original data blocks to obtain the global check block of the i-th group, where the value of i ranges from 1 to the number of the groups.

4. The method according to claim 2, wherein The generating local check blocks for each group based on the Cauchy matrix, the multiple original data blocks, a preset cross block coefficient, and the cross blocks includes: summing the data of each row of the Cauchy matrix by column to obtain the Cauchy matrix sum values corresponding to each column, and the Cauchy matrix sum values corresponding to each column correspond to the respective original data blocks; for the i-th group, determining the local check block of the i-th group in the following manner, where the value of i ranges from 1 to the number of the groups: multiplying the i-th row of the Cauchy matrix by the vector formed by the multiple original data blocks, adding the sum of the Cauchy matrix sum values corresponding to the i-th group multiplied by the sum of the respective target original data blocks corresponding to the i-th group, and adding the sum of the i-th cross block coefficient multiplied by the sum of the cross blocks to obtain the local check block of the i-th group; where the target original data blocks corresponding to the i-th group are part or all of the original data blocks included in the i-th group, the target original data blocks corresponding to the multiple groups include all the original data blocks and the target original data blocks corresponding to each group do not repeat; the Cauchy matrix sum values corresponding to the i-th group refer to the Cauchy matrix sum values corresponding to the respective target original data blocks corresponding to the i-th group; and the sum of the cross block coefficients is zero.

5. A data block recovery method, including: Determine the missing target data block; wherein, The target data block is a data block in a group obtained by encoding multiple original data blocks by using the method according to any one of claims 1-4; If the target data block is any cross block, then use the data blocks in any group to which the cross block belongs to recover the target data block.

6. The method according to claim 5 further comprises: When recovering the target data block using the data blocks in any one of the groups to which the cross block belongs, if the recovery is not completed within a preset time, then the data blocks in other groups to which the cross block belongs are used again to recover the target data block.

7. The method according to claim 5, further comprising: If the target data block is any data block other than the cross block, then the target data block is recovered based on the data blocks in the group to which the target data block belongs.

8. The method according to claim 5, further comprising: If the target data block is any n data blocks, where n ranges from 2 to the number of groups plus 1, then the target data block is recovered based on the data blocks in each group.

9. A data block encoding device, comprising: A grouping module, configured to divide a plurality of original data blocks into a plurality of groups, where at least one cross block is included in the plurality of groups, and the cross block is the same original data block in the plurality of groups; A construction module, configured to construct a Cauchy matrix based on the number of the original data blocks and the number of the groups; An encoding module, configured to generate a global check block for each group based on the Cauchy matrix and the plurality of original data blocks, generate a local check block for each group based on the Cauchy matrix, the plurality of original data blocks, a preset cross block coefficient, and the cross block, and add the global check block and the local check block of each group to each group, where the sum of the global check blocks of the plurality of groups constitutes a new global check block.

10. A data block recovery device, comprising: A determination module, configured to determine a lost target data block; where the target data block is a data block in a group obtained by encoding a plurality of original data blocks using the method according to any one of claims 1-4; a recovery module, configured to, if the target data block is any cross block, then recover the target data block using the data blocks in any one of the groups to which the cross block belongs.

11. An electronic device, comprising: A memory and a processor; The memory is used to store a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program runs, the processor executes the method according to any one of claims 1-8. 18 12. A computer-readable storage medium, wherein, A computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the processor executes the method according to any one of claims 1-8.

13. A computer program product comprising a computer program, wherein, When the computer program is executed by the processor, the method according to any one of claims 1-8 is implemented.

Images