Erasure code coding method and device with low repair bandwidth

Abstract

The invention belongs to the technical field of computer storage, and particularly discloses an erasure code coding method and device with low repair bandwidth. According to the application, k data blocks and m verification blocks form a strip, in a distributed storage system using an erasure code coding mode, the strip is striped, and the verification information of the sub data blocks in a part of sub-strips is embedded into the sub verification blocks of other sub-strips, so that the data blocks in the sub-strips can be verified, and the reliability of the storage system is improved. Therefore, when the storage nodes in the distributed storage system fail, the repair bandwidth of part of the sub-strips can be reduced through the embedded verification information.

Description

Technical Field

[0001] The present application belongs to the field of computer storage technology, and more specifically, to a method and device for encoding erasure codes with low repair bandwidth. Background Art

[0002] Distributed storage systems enable enterprises to manage large amounts of data across multiple storage nodes. Distributed storage systems are usually composed of cheap commercial servers, so they have a high failure rate, so they often prevent failures by increasing data redundancy. Erasure coding is favored by many distributed storage systems because it can provide the same fault tolerance with less storage overhead. Erasure coding technology divides the original data into several data blocks, and then generates several check blocks through specific matrix operations. These data blocks and check blocks form a stripe.

[0003] However, while erasure coding provides low-cost redundancy, it incurs high repair overhead.

[0004] For example, for an erasure code of (n, k) = (14, 10), n is the total number of data blocks and check blocks included in the stripe, and k is the number of data blocks. The original data is divided into 10 data blocks, and then 4 additional check blocks are generated through erasure coding, and these 10 data blocks and 4 check blocks are stored on different storage nodes of the distributed storage system. If any data block fails due to a fault, the storage node performing the repair needs to repair the failed block by requesting and decoding 10 data blocks from the other 10 available storage nodes, which results in a 10-fold bandwidth amplification.

[0005] Too high a repair bandwidth will affect the front-end storage service of the storage system, and may also prolong the repair time, causing the system to be in a degraded state for a long time, further increasing the risk of data loss. Therefore, how to solve the problem of too high a repair bandwidth when a storage node fails in a distributed storage system using erasure coding is an urgent problem to be solved. Summary of the invention

[0006] In view of the defects of the prior art, the purpose of the present application is to provide a method and device for erasure coding with low repair bandwidth, aiming to solve the problem of excessively high repair bandwidth in the case of storage node failure in a distributed storage system using erasure coding in the related art.

[0007] To achieve the above objectives, in a first aspect, the present application provides an erasure coding method with low repair bandwidth, comprising:

[0008] Divide a stripe containing k data blocks and m check blocks into t sub-stripes, each sub-strip includes k sub-data blocks and m sub-check blocks, t=m;

[0009] Group the k sub-data blocks in each sub-strip;

[0010] The check information is calculated according to the grouping, and is embedded into the sub-check blocks in the corresponding sub-strip.

[0011] In some embodiments, a stripe including k data blocks and m check blocks is divided into t sub-stripes, including:

[0012] Divide each data block into t sub-data blocks;

[0013] Divide each check block into t sub-check blocks;

[0014] Any sub-stripe is determined according to any sub-data block in each data block and any sub-check block in each check block.

[0015] In some embodiments, grouping the k sub-data blocks in each sub-strip includes:

[0016] For the i-th sub-strip among t sub-strips, 0 <i≤t:

[0017] The k sub-data blocks in the i-th sub-strip are divided into i′ groups, i′=i.

[0018] In some embodiments, the k sub-data blocks in the i-th sub-strip are divided into i′ groups, including:

[0019] When k is divisible by i′, the k sub-data blocks in the i-th sub-strip are divided into i′ groups;

[0020] If k is not divisible by i′, the k sub-data blocks in the i-th sub-strip are divided into i′ groups;

[0021] Among them, the first group to the (i′-1)th group include sub-data blocks, the i′th group includes sub-blocks.

[0022] In some embodiments, calculating the check information according to the grouping and embedding the check information into the sub-check blocks in the corresponding sub-strip includes:

[0023] For the jth packet in the i-th sub-strip, 1≤j≤i′, 0 <i≤t:

[0024] Obtain (t-i+1) linearly independent check information, and embed the (t-i+1) linearly independent check information into the γth sub-check block in the jth sub-strip in sequence, i≤γ≤m;

[0025] For the i″th sub-strip, 1 <i″≤t:

[0026] Obtain (i″-1) linearly independent check information, and embed the (i″-1) linearly independent check information into the γ′th sub-check block in the i″th sub-strip, 1≤γ′≤i″-1;

[0027] For the i′′′′th sub-strip, 1≤i′′′ <t:

[0028] Obtain t linearly independent verification information;

[0029] The t linearly independent check information are respectively negated, and the t linearly independent check information after being negated are embedded into the m sub-check blocks in the (i″′+1)th sub-strip.

[0030] In some embodiments, a stripe including k data blocks and m check blocks is obtained as follows:

[0031] Divide the original data into k data blocks;

[0032] Based on the erasure code encoding method, k data blocks are encoded to generate m check blocks;

[0033] A stripe is obtained based on k data blocks and m check blocks.

[0034] In a second aspect, the present application provides an erasure coding device with low repair bandwidth, comprising:

[0035] An acquisition module is used to divide a stripe including k data blocks and m check blocks into t sub-stripes, each sub-strip includes k sub-data blocks and m sub-check blocks, and t=m;

[0036] A grouping module, used for grouping k sub-data blocks in each sub-strip;

[0037] The encoding module is used to calculate the check information according to the grouping and embed the check information into the sub-check block in the corresponding sub-strip.

[0038] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the programs stored in the memory are executed, the processor is used to execute the low repair bandwidth erasure code encoding method described in the first aspect or any embodiments of the first aspect.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the low repair bandwidth erasure code encoding method described in the first aspect or any embodiments of the first aspect.

[0040] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the erasure code encoding method with low repair bandwidth described in the first aspect or any embodiments of the first aspect.

[0041] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0042] The present application provides a method and device for encoding erasure codes with low repair bandwidth. By forming a stripe with k data blocks and m check blocks, in a distributed storage system using erasure code encoding, the stripe is sub-striped, and the check information of sub-data blocks in some sub-stripes is embedded into sub-check blocks of other sub-stripes. When a storage node in a distributed storage system fails, the repair bandwidth of some sub-stripes can be reduced by using these embedded check information. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of a method for encoding erasure codes with low repair bandwidth provided by an embodiment of the present application;

[0044] Figure 2 It is a schematic diagram of the code generation process provided by an embodiment of the present application;

[0045] Figure 3 It is a structural schematic diagram of a low repair bandwidth erasure code encoding device provided in an embodiment of the present application;

[0046] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0049] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0050] In the description of the embodiments of the present application, unless otherwise specified, “plurality” means two or more than two.

[0051] In a distributed storage system, an erasure code with parameters (n, k), where n = k + m, that is, the original data O of size M is divided into k equal data blocks O1, ..., O k , each data block contains M / k data. Then, m check blocks are generated through certain regular coding. By placing the total n coding blocks separately in different storage nodes of the distributed storage system, it is guaranteed that the original data O can be restored by any k′ (k′>k) coding blocks. If k′=k, the erasure code is called Maximum Distance Separable (MDS).

[0052] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0053] See also Figure 1 , an embodiment of the present application provides a low repair bandwidth erasure code encoding method, which may include: step 110, step 120 and step 130.

[0054] Step 110: divide the stripe including k data blocks and m check blocks into t sub-stripes, each sub-strip includes k sub-data blocks and m sub-check blocks, t=m;

[0055] Step 120 groups the k sub-data blocks in each sub-strip;

[0056] Step 130 calculates check information according to the grouping, and embeds the check information into the sub-check blocks in the corresponding sub-strip.

[0057] In the embodiment of the present application, striping is a data storage technology that divides continuous data into data blocks of the same size, and then writes each piece of data to different disks in the array. In erasure coding, a stripe contains k data blocks and m check blocks generated by encoding the k data blocks. Wherein, k and m are both integers greater than 1.

[0058] For a stripe obtained by erasure coding, it has k data blocks and m check blocks. Assume that the size of the data block and the check block are both M′. It is required that the data block and the check block can be divided into t parts, that is, M′ / t. For the (k+m) coding blocks in a stripe, these blocks are divided into t sub-blocks, then t sub-stripes are generated, each of which contains k sub-data blocks and m sub-check blocks. Among them, the value of t can be the same as the value of m.

[0059] The k sub-data blocks in each sub-strip are grouped. During the grouping, it is necessary to ensure that the k sub-data blocks can be grouped evenly as much as possible.

[0060] According to the grouping, the check information of the sub-data block and the check information of the sub-strip are calculated, and the check information is embedded into the sub-check block in the corresponding sub-strip.

[0061] An embodiment of the present application provides a method for encoding erasure codes with low repair bandwidth. By forming a stripe with k data blocks and m check blocks, in a distributed storage system using erasure code encoding, the stripe is sub-striped, and the check information of sub-data blocks in some sub-stripes is embedded into sub-check blocks of other sub-strips. When a storage node in a distributed storage system fails, the repair bandwidth of some sub-stripes can be reduced by using these embedded check information.

[0062] Further, in some embodiments, a stripe including k data blocks and m check blocks is obtained by:

[0063] Divide the original data into k data blocks;

[0064] Based on the erasure code encoding method, k data blocks are encoded to generate m check blocks;

[0065] A stripe is obtained based on k data blocks and m check blocks.

[0066] In the embodiment of the present application, an original data is evenly divided into k data blocks, and m check blocks are generated by using the k data blocks according to a selected erasure coding scheme as a basic code. The k data blocks and the m check blocks form a stripe.

[0067] Further, in some embodiments, in step 110, dividing the stripe including the k data blocks and the m check blocks into t sub-stripes may include:

[0068] Divide each data block into t sub-data blocks;

[0069] Divide each check block into t sub-check blocks;

[0070] Determine any sub-strip according to any sub-data block in each data block and any sub-parity block in each parity block.

[0071] In a specific implementation, for a strip with k data blocks and m parity blocks, assume that the size of each data block and each parity block is M′, and each data block and each parity block can be divided into t parts.

[0072] By dividing each data block into t sub-data blocks and each parity block into t sub-parity blocks, kt sub-data blocks and mt sub-parity blocks are obtained.

[0073] For any sub-strip among the t sub-strips, it can be obtained by arbitrarily selecting k sub-data blocks from the kt sub-data blocks and arbitrarily selecting m sub-parity blocks from the mt sub-parity blocks.

[0074] Further, in some embodiments, in step 120, grouping the k sub-data blocks in each sub-strip may include:

[0075] For the i-th sub-strip among the t sub-strips, 0 < i ≤ t:

[0076] Divide the k sub-data blocks in the i-th sub-strip into i′ groups, where i′ = i.

[0077] In the embodiments of the present application, for any sub-strip among the t sub-strips (assuming the i-th sub-strip, 0 < i ≤ t), the following grouping process is performed:

[0078] Divide the k sub-data blocks in the i-th sub-strip into i′ groups, and the number of groups is determined by the value of i. In the embodiments of the present application, i′ = i.

[0079] Exemplarily, assume there are 3 sub-strips, and each sub-strip has 6 sub-data blocks. Then the number of groups of sub-data blocks in the first sub-strip is 1, the number of groups of sub-data blocks in the second sub-strip is 2, and the number of sub-data blocks in the third sub-strip is 3.

[0080] Further, in some embodiments, in the above steps, dividing the k sub-data blocks in the i-th sub-strip into i′ groups may include:

[0081] In the case where k is divisible by i′, divide the k sub-data blocks in the i-th sub-strip into i′ equal groups;

[0082] In the case where k is not divisible by i′, divide the k sub-data blocks in the i-th sub-strip into i′ groups;

[0083] Among them, the first group to the (i′ - 1)-th group include sub-data blocks, the i′th group includes sub-blocks.

[0084] In a specific implementation, when the k sub-data blocks in the i-th sub-strip are divided into i′ groups, there are the following two situations:

[0085] 1. If k is divisible by i′, the k sub-data blocks in the i-th sub-strip can be evenly divided into i′ groups, and the first group to the i′-1 group include sub-data blocks, the i′th group includes Sub-data blocks, Representatives Round up. Since k is divisible by i′ at this time, mod(k, i′)=0, that is, the number of sub-data blocks included in the i′th group is the same as the number of sub-data blocks included in the first group to the i′-1th group.

[0086] 2. When k is not divisible by i′, the k sub-data blocks in the ith sub-strip are still divided into i′ groups. Since k is not divisible by i′, mod(k,i′)≠0, that is, the k sub-data blocks in the ith sub-strip cannot be evenly divided into i′ groups. The number of sub-data blocks in the first group to the i′-1th group is the same, which is The i′th group includes sub-data blocks, that is, all the redundant sub-data blocks are placed in the i′th group.

[0087] Further, in some embodiments, in step 130, calculating the check information according to the grouping and embedding the check information into the sub-check block in the corresponding sub-strip may include:

[0088] For the jth packet in the i-th sub-strip, 1≤j≤i′, 0 <i≤t:

[0089] Obtain (t-i+1) linearly independent check information, and embed the (t-i+1) linearly independent check information into the γth sub-check block in the jth sub-strip in sequence, i≤γ≤m;

[0090] For the i″th sub-strip, 1 <i″≤t:

[0091] Obtain (i″-1) linearly independent check information, and embed the (i″-1) linearly independent check information into the γ′th sub-check block in the i″th sub-strip, 1≤γ′≤i″-1;

[0092] For the i′′′′th sub-strip, 1≤i′′′ <t:

[0093] Obtain t linearly independent parity information;

[0094] Negate the t linearly independent parity information respectively, and embed the t negated linearly independent parity information into m sub-parity blocks in the (i″′ + 1)-th sub-strip.

[0095] In a specific implementation, for the sub-data block in the j-th group of the i-th sub-strip, where 1 ≤ j ≤ i′, 0 < i ≤ t, calculate (t - i + 1) linearly independent parity information based on RS coding, and embed these parity information into the γ-th parity sub-block of the j-th sub-strip in turn (1 ≤ γ ≤ m).

[0096] For the i″-th sub-strip, 1 < i″ ≤ t, calculate (i″ - 1) linearly independent parity information about this sub-strip based on RS coding, and embed these parity information into the γ′-th sub-parity block in the i″-th sub-strip, 1 ≤ γ′ ≤ i″ - 1.

[0097] For the i″′-th sub-strip, 1 ≤ i″′ < t, calculate t linearly independent parity information based on RS coding, negate these parity information respectively, and embed the t negated linearly independent parity information into m sub-parity blocks in the (i″′ + 1)-th sub-strip.

[0098] Exemplarily, refer to Figure 2 , in the embodiment of the present application, the erasure code parameters are (n, k) = (9, 6).

[0099] As Figure 2 shown in (1) of [], taking the RS code as the basic code, divide an original data of size 6M into 6 data blocks O1,..., O6 in total, and through RS coding, generate 3 linearly independent parity blocks, assumed to be f(O), g(O), h(O) respectively. These 6 data blocks and 3 parity blocks form a strip, where the size of each data block and parity block is M′.

[0100] As Figure 2 shown in (2) of [], divide a strip generated by RS coding into 3 sub-strips a, b, c in total, each sub-strip contains 6 sub-data blocks and 3 sub-parity blocks, and the size of each sub-data block and sub-parity block is M′ / 3.

[0101] As Figure 2As shown in (3), the sub-data blocks in each sub-strip are grouped in sequence. Assume that the first sub-strip a is divided into 1 group, with 6 data blocks in each group, denoted as {a1, a2,..., a6}; the second strip b is divided into 2 groups, with 3 sub-data blocks in each group, namely {b1, b2, b3} and {b4, b5, b6}; the third sub-strip c is divided into 3 groups, with 2 sub-data blocks in each group, namely {c1, c2}, {c3, c4}, and {c5, c6}.

[0102] As Figure 2 shown in (4), for the sub-data block in the j-th group of the i-th sub-strip, where 1 ≤ j ≤ i′, calculate (t - i + 1) linearly independent parity-check information, and embed this parity-check information into the γ-th parity-check sub-block of the j-th sub-strip in sequence, where 1 ≤ γ ≤ m.

[0103] Specifically, calculate 3 linearly independent parity-check information of the first sub-strip a, denoted as f(a), g(a), and h(a), based on RS coding, and embed them into the 3 sub-parity-check blocks of the first sub-strip a respectively; calculate 2 linearly independent parity-check information of the sub-data blocks in the first group of the second sub-strip b (assumed to be {b1, b2, b3}), denoted as g(b 123 ), and h(b 123 ), and embed them into the last two sub-parity-check blocks of the first sub-strip a respectively. For example, embed g(b 123 ) into the second sub-parity-check block of the first sub-strip a and embed h(b 123 ) into the third sub-parity-check block of the first sub-strip a. Similarly, embed the 2 linearly independent parity-check information (assumed to be g(b 456 ) and h(b 456 )) of the sub-data blocks in the second group of the second sub-strip b into the last 2 parity-check sub-blocks of the second sub-strip b respectively; calculate the parity-check information of the sub-data blocks in the 3 groups of the third sub-strip c (assumed to be {h(c 12 ), h(c 34 ), h(c 56 )}), and embed them into the last 1 sub-parity-check block of the third sub-strip c.

[0104] As Figure 2 shown in (5), for the i″-th sub-strip, where 1 < i″ ≤ t, calculate (i″ - 1) linearly independent parity-check information about this sub-strip, and embed it into the γ′-th sub-parity-check block in the i″-th sub-strip, where 1 ≤ γ′ ≤ i″ - 1.

[0105] Specifically, calculate 1 parity information (assumed to be f(b)) of the second sub-strip b and embed it into the first parity block of sub-strip b; calculate 2 linearly independent parity information (f(c), g(c)) of the third sub-strip c and embed them into the first 2 sub-parity blocks of sub-strip c respectively.

[0106] As Figure 2 shown in (6) therein, for the i″′-th sub-strip, 1 ≤ i″′ < t, calculate t linearly independent parity information based on RS coding, take the negative values of the t linearly independent parity information respectively, and embed the t linearly independent parity information after taking the negative values into m sub-parity blocks in the (i″′ + 1)-th sub-strip.

[0107] Specifically, calculate 3 linearly independent parity information (assumed to be f(a), g(a), h(a)) of the first sub-strip a, take the negative values to get -f(a), -g(a), -h(a), and embed them into 3 sub-parity blocks of the second sub-strip b; similarly, calculate 3 linearly independent parity information (assumed to be f(b), g(b), h(b)) of the second sub-strip b, take the negative values to get -f(b), -g(b), -h(b), and embed them into 3 sub-parity blocks of the third sub-strip c respectively.

[0108] The encoding scheme generated in this way can effectively reduce the repair bandwidth when a single storage node fails in a distributed storage system using erasure codes.

[0109] Taking the failure of the information (assumed to be {a1, b1, c1}) in the first storage node as an example. Read the data {a2,..., a6, f(a)} from other storage nodes. According to the MDS property of the underlying RS coding, a1 can be recovered. Based on the existing information {a1, a2,..., a6}, calculate g(a); read the second sub-parity block g(a) + g(b 123 ) of sub-strip a and cancel the known g(a) to get g(b 123 ); read {b2, b3} and recover b1 according to g(b 123 ). Read the third sub-parity block h(a) + h(b 123 ) + h(c 12 ) of sub-strip a, calculate h(a) and h(b 123 ) according to the known information, cancel them from this sub-parity block to get h(c 12 ), then read c2 and recover c1, thus recovering all the failed information in node 1. In this process, a total of 11 sub-blocks are read and transmitted in the network. Compared with the underlying RS coding that needs to read and transmit 18 sub-blocks, the repair bandwidth is reduced by 39%.

[0110] An embodiment of the present application provides a method for encoding erasure codes with low repair bandwidth. When repairing failed blocks of erasure codes, the failed data block information can be directly solved through the embedded verification information of a specific sub-data block group to complete the repair operation, thereby avoiding reading too much sub-block information from other nodes and reducing the repair bandwidth of the distributed storage system.

[0111] A low repair bandwidth erasure code encoding device provided by the present invention is described below. The low repair bandwidth erasure code encoding device described below and the low repair bandwidth erasure code encoding method described above can correspond to each other.

[0112] See also Figure 3 , an embodiment of the present application provides a low repair bandwidth erasure code encoding device, which may include: an acquisition module 310, a grouping module 320 and an encoding module 330.

[0113] The acquisition module 310 is used to divide the stripe including k data blocks and m check blocks into t sub-stripes, each sub-strip includes k sub-data blocks and m sub-check blocks, and t=m;

[0114] A grouping module 320, configured to group the k sub-data blocks in each sub-strip;

[0115] The encoding module 330 is used to calculate the check information according to the grouping, and embed the check information into the sub-check block in the corresponding sub-strip.

[0116] An embodiment of the present application provides an erasure code encoding device with low repair bandwidth. By forming a stripe with k data blocks and m check blocks, in a distributed storage system using an erasure code encoding method, the stripe is sub-striped and the check information of the sub-data blocks in some sub-stripes is embedded into the sub-check blocks of other sub-stripes. Therefore, when a storage node in a distributed storage system fails, the repair bandwidth of some sub-stripes can be reduced by using these embedded check information.

[0117] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0118] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0119] Based on the method in the above embodiment, the present application embodiment provides an electronic device, see Figure 4The electronic device may include: a processor (Processor) 410, a communication interface (Communications Interface) 420, a memory (Memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the method in the above embodiment.

[0120] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0121] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0122] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0123] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0124] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0125] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0126] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0127] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for encoding erasure codes with low repair bandwidth, characterized in that: include: Divide a stripe containing k data blocks and m check blocks into t sub-stripes, each sub-strip includes k sub-data blocks and m sub-check blocks, t=m; Group the k sub-data blocks in each sub-strip; The check information is calculated according to the grouping, and the check information is embedded into the sub-check blocks in the corresponding sub-strip.

2. The low repair bandwidth erasure coding method according to claim 1, characterized in that: The step of dividing a stripe including k data blocks and m check blocks into t sub-stripes comprises: Divide each data block into t sub-data blocks; Divide each check block into t sub-check blocks; Any sub-stripe is determined according to any sub-data block in each data block and any sub-check block in each check block.

3. The low repair bandwidth erasure coding method according to claim 1, characterized in that: The grouping of the k sub-data blocks in each sub-strip includes: For the i-th substrip among t substrips, 0 <i≤t: The k sub-data blocks in the i-th sub-strip are divided into i′ groups, i′=i.

4. The low repair bandwidth erasure coding method according to claim 3, characterized in that: The step of dividing the k sub-data blocks in the i-th sub-strip into i′ groups comprises: When k is divisible by i′, the k sub-data blocks in the i-th sub-strip are divided into i′ groups; If k is not divisible by i′, the k sub-data blocks in the i-th sub-strip are divided into i′ groups; Among them, the first group to the (i′-1)th group include sub-data blocks, the i′th group includes sub-blocks.

5. The low repair bandwidth erasure coding method according to claim 1, characterized in that: The step of calculating the check information according to the grouping and embedding the check information into the sub-check blocks in the corresponding sub-strip includes: For the jth packet in the i-th sub-strip, 1≤j≤i′, 0 <i≤t: Obtain (t-i+1) linearly independent check information, and embed the (t-i+1) linearly independent check information into the γth sub-check block in the jth sub-strip in sequence, i≤γ≤m; For the i″th sub-strip, 1 <i″≤t: Obtain (i″-1) linearly independent check information, and embed the (i″-1) linearly independent check information into the γ′th sub-check block in the i″th sub-strip, 1≤γ′≤i″-1; For the i′′′′th sub-strip, 1≤i′′′ <t: Obtain t linearly independent verification information; The t linearly independent check information are respectively negated, and the t linearly independent check information after being negated are embedded into the m sub-check blocks in the (i″′+1)th sub-strip.

6. The low repair bandwidth erasure coding method according to any one of claims 1 to 5, characterized in that: The stripe including k data blocks and m check blocks is obtained in the following manner: Divide the original data into k data blocks; Based on the erasure code encoding method, the k data blocks are encoded to generate m check blocks; The stripe is obtained according to the k data blocks and the m check blocks.

7. A low repair bandwidth erasure code encoding device, characterized in that: include: An acquisition module is used to divide a stripe including k data blocks and m check blocks into t sub-stripes, each sub-strip includes k sub-data blocks and m sub-check blocks, and t=m; A grouping module, used for grouping k sub-data blocks in each sub-strip; The encoding module is used to calculate the check information according to the grouping, and embed the check information into the sub-check block in the corresponding sub-strip.

8. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the low repair bandwidth erasure code encoding method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is enabled to execute the erasure coding method with low repair bandwidth as described in any one of claims 1-6.

10. A computer program product, characterized in that When the computer program product runs on a processor, the processor is enabled to execute the erasure coding method with low repair bandwidth as described in any one of claims 1-6.

Images