A low-repair-bandwidth erasure code encoding method and device

Abstract

The application belongs to the technical field of computer storage, and particularly discloses a low-repair-bandwidth erasure coding method and device. According to the application, k data blocks and m check blocks are combined into a stripe, the stripe is sub-stripped, and the check information of the sub-data blocks in some sub-stripe is embedded into the sub-check blocks of other sub-stripe, so that when a storage node in a distributed storage system fails, the repair bandwidth of some sub-stripe can be reduced through the embedded check information.

Description

Technical Field

[0001] The present application belongs to the field of computer storage technology, and more specifically, relates to a method and device for encoding erasure codes with low repair bandwidth. Background Art

[0002] Distributed storage systems enable enterprises to manage large amounts of data across multiple storage nodes. However, distributed storage systems are typically composed of inexpensive commodity servers, resulting in a high failure rate. Therefore, data redundancy is often employed to mitigate these failures. Erasure coding is favored by many distributed storage systems because it offers the same fault tolerance with less storage overhead. Erasure coding technology divides the original data into several data blocks, then generates several parity blocks through specialized matrix operations. These data blocks and parity blocks form a stripe.

[0003] However, while erasure coding provides low-cost redundancy, it incurs high repair overhead.

[0004] For example, for an erasure code with (n,k)=(14,10), where n is the total number of data blocks and parity blocks in the stripe, and k is the number of data blocks, the original data is divided into 10 data blocks. Four additional parity blocks are then generated through erasure coding, and these 10 data blocks and four parity blocks are stored on different storage nodes in the distributed storage system. If any data block fails, the repairing storage node must request and decode 10 data blocks from the other 10 available storage nodes to repair the failed block, resulting in a 10-fold bandwidth amplification.

[0005] Excessive repair bandwidth can affect the storage system's front-end storage services and potentially prolong repair times, leaving the system in a degraded state for extended periods, further increasing the risk of data loss. Therefore, addressing the high repair bandwidth associated with storage node failures in distributed storage systems using erasure coding is a pressing issue. Summary of the Invention

[0006] In response to the defects of the existing technology, the purpose of this application is to provide a low-repair-bandwidth erasure coding encoding method and device, aiming to solve the problem of excessively high repair bandwidth in the case of storage node failure in a distributed storage system using erasure coding in the related technology.

[0007] To achieve the above objectives, in a first aspect, the present application provides a method for encoding erasure codes with low repair bandwidth, comprising:

[0008] Divide a stripe containing k data blocks and m parity blocks into t sub-stripes, each sub-strip contains k sub-data blocks and m sub-parity blocks, where t = m;

[0009] Group the k sub-data blocks in each sub-strip;

[0010] The check information is calculated according to the grouping and embedded into the sub-check blocks in the corresponding sub-strip.

[0011] In some embodiments, a stripe including k data blocks and m parity blocks is divided into t sub-stripes, including:

[0012] Divide each data block into t sub-data blocks;

[0013] Divide each check block into t sub-check blocks;

[0014] Any sub-stripe is determined according to any sub-data block in each data block and any sub-check block in each check block.

[0015] In some embodiments, grouping the k sub-data blocks in each sub-strip includes:

[0016] For the i-th sub-strip among t sub-strips, 0 <i≤t:

[0017] The k sub-data blocks in the i-th sub-strip are divided into i′ groups, where i′=i.

[0018] In some embodiments, the k sub-data blocks in the i-th sub-strip are divided into i′ groups, including:

[0019] If k is divisible by i′, the k sub-data blocks in the i-th sub-strip are divided into i′ groups;

[0020] If k is not divisible by i′, the k sub-data blocks in the i-th sub-strip are divided into i′ groups;

[0021] Among them, the first group to the (i′-1)th group include sub-data blocks, the i′th group includes sub-data blocks.

[0022] In some embodiments, calculating parity information according to the grouping and embedding the parity information into the sub-parity blocks in the corresponding sub-strip includes:

[0023] For the jth packet in the i-th substrip, 1≤j≤i′, 0 <i≤t:

[0024] Obtain (t-i+1) linearly independent check information, and sequentially embed the (t-i+1) linearly independent check information into the γth sub-check block in the jth sub-strip, where i≤γ≤m;

[0025] For the i″th sub-strip, 1 <i″≤t:

[0026] Obtain (i″-1) linearly independent check information, and embed the (i″-1) linearly independent check information into the γ′-th sub-check block in the i″-th sub-strip, 1≤γ′≤i″-1;

[0027] For the i′′′th sub-strip, 1≤i′′ <t:

[0028] Obtain t linearly independent verification information;

[0029] The t linearly independent check information are respectively negated, and the t linearly independent check information after being negated are embedded into the m sub-check blocks in the (i″′+1)th sub-strip.

[0030] In some embodiments, a stripe containing k data blocks and m parity blocks is obtained as follows:

[0031] Divide the original data into k data blocks;

[0032] Based on the erasure code encoding method, k data blocks are encoded to generate m check blocks;

[0033] Get a stripe based on k data blocks and m check blocks.

[0034] In a second aspect, the present application provides an erasure coding apparatus with low repair bandwidth, comprising:

[0035] An acquisition module is configured to divide a stripe including k data blocks and m check blocks into t sub-stripes, each sub-strip including k sub-data blocks and m sub-check blocks, where t=m;

[0036] A grouping module, configured to group k sub-data blocks in each sub-strip;

[0037] The encoding module is configured to calculate check information according to the grouping and embed the check information into the sub-check blocks in the corresponding sub-strip.

[0038] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the programs stored in the memory are executed, the processor is used to execute the low repair bandwidth erasure code encoding method described in the first aspect or any embodiments of the first aspect.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the low repair bandwidth erasure code encoding method described in the first aspect or any embodiment of the first aspect.

[0040] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the low repair bandwidth erasure coding method described in the first aspect or any embodiments of the first aspect.

[0041] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the existing technologies:

[0042] The present application provides a method and apparatus for encoding erasure codes with low repair bandwidth. By forming a stripe with k data blocks and m check blocks, in a distributed storage system using erasure code encoding, the stripe is sub-striped, and the check information of the sub-data blocks in some sub-stripes is embedded into the sub-check blocks of other sub-stripes. Thus, when a storage node in a distributed storage system fails, the repair bandwidth of some sub-stripes can be reduced by using the embedded check information. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of a method for encoding erasure codes with low repair bandwidth provided by an embodiment of the present application;

[0044] Figure 2 This is a schematic diagram of the code generation process provided by an embodiment of the present application;

[0045] Figure 3 1 is a schematic structural diagram of a low-repair-bandwidth erasure code encoding device provided in an embodiment of the present application;

[0046] Figure 4 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0048] The term "and / or" as used herein describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The symbol " / " as used herein indicates that the related objects are in an "or" relationship, for example, A / B means either A or B.

[0049] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0050] In the description of the embodiments of the present application, unless otherwise specified, “plurality” means two or more.

[0051] In a distributed storage system, an erasure code with parameters (n, k), where n = k + m, divides the original data O of size M into k equal data blocks O1, ..., O k Each data block contains M / k pieces of data. Then, m check blocks are generated using a certain encoding rule. By separating these n code blocks and placing them in different storage nodes of the distributed storage system, the original data O can be restored from any k′ (k′>k) code blocks. If k′=k, the erasure code is called a Maximum Distance Separable (MDS) code.

[0052] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0053] See also Figure 1 , an embodiment of the present application provides a low repair bandwidth erasure code encoding method, which may include: step 110, step 120 and step 130.

[0054] Step 110: divide the stripe containing k data blocks and m check blocks into t sub-stripes, each sub-strip includes k sub-data blocks and m sub-check blocks, where t=m;

[0055] Step 120 groups the k sub-data blocks in each sub-strip;

[0056] Step 130 calculates check information according to the grouping and embeds the check information into the sub-check blocks in the corresponding sub-strip.

[0057] In the embodiments of this application, striping is a data storage technology that divides continuous data into blocks of equal size and then writes each block of data to different disks in the array. In erasure coding, a stripe consists of k data blocks and m parity blocks generated by encoding these k data blocks. Both k and m are integers greater than 1.

[0058] For a stripe generated using erasure coding, it has k data blocks and m parity blocks. Assume that the size of the data blocks and parity blocks is M′. The data blocks and parity blocks must be divided equally into t parts, that is, M′ / t. For the (k+m) coding blocks in a stripe, these blocks are divided equally into t sub-blocks, resulting in t sub-stripe, each of which contains k sub-data blocks and m sub-parity blocks. The value of t can be the same as the value of m.

[0059] The k sub-data blocks in each sub-strip are grouped. During the grouping, it is necessary to ensure that the k sub-data blocks can be grouped evenly.

[0060] According to the grouping, the parity information of the sub-data block and the parity information of the sub-strip are calculated, and these parity information are embedded into the sub-parity blocks in the corresponding sub-strip.

[0061] An embodiment of the present application provides a low repair bandwidth erasure code encoding method, which combines k data blocks and m check blocks into a stripe. In a distributed storage system using an erasure code encoding method, the stripe is sub-striped and the check information of the sub-data blocks in some sub-stripes is embedded into the sub-check blocks of other sub-stripes. When a storage node in a distributed storage system fails, the repair bandwidth of some sub-stripes can be reduced by using these embedded check information.

[0062] Furthermore, in some embodiments, a stripe including k data blocks and m parity blocks is obtained as follows:

[0063] Divide the original data into k data blocks;

[0064] Based on the erasure code encoding method, k data blocks are encoded to generate m check blocks;

[0065] Get a stripe based on k data blocks and m check blocks.

[0066] In this embodiment of the present application, an original data is evenly divided into k data blocks, and m check blocks are generated from these k data blocks according to a selected erasure coding scheme as a basic code. These k data blocks and the m check blocks form a stripe.

[0067] Furthermore, in some embodiments, in step 110, dividing the stripe including k data blocks and m parity blocks into t sub-stripes may include:

[0068] Divide each data block into t sub-data blocks;

[0069] Divide each check block into t sub-check blocks;

[0070] Determine any sub-strip according to any sub-data block in each data block and any sub-parity block in each parity block.

[0071] In a specific implementation, for a strip with k data blocks and m parity blocks, assume that the size of each data block and each parity block is M′, and each data block and each parity block can be divided into t parts.

[0072] By dividing each data block into t sub-data blocks and each parity block into t sub-parity blocks, kt sub-data blocks and mt sub-parity blocks are obtained.

[0073] For any sub-strip in the t sub-strips, it can be obtained by arbitrarily selecting k sub-data blocks from the kt sub-data blocks and arbitrarily selecting m sub-parity blocks from the mt sub-parity blocks.

[0074] Further, in some embodiments, in step 120, grouping the k sub-data blocks in each sub-strip may include:

[0075] For the i-th sub-strip in the t sub-strips, 0 < i ≤ t:

[0076] Divide the k sub-data blocks in the i-th sub-strip into i′ groups, where i′ = i.

[0077] In the embodiments of the present application, for any sub-strip in the t sub-strips (assuming the i-th sub-strip, 0 < i ≤ t), the following grouping process is performed:

[0078] Divide the k sub-data blocks in the i-th sub-strip into i′ groups, and the number of groups is determined by the value of i. In the embodiments of the present application, i′ = i.

[0079] Exemplarily, assume there are 3 sub-strips and each sub-strip has 6 sub-data blocks. Then the number of groups of the sub-data blocks in the first sub-strip is 1, the number of groups of the sub-data blocks in the second sub-strip is 2, and the number of sub-data blocks in the third sub-strip is 3.

[0080] Further, in some embodiments, in the above steps, dividing the k sub-data blocks in the i-th sub-strip into i′ groups may include:

[0081] When k is divisible by i′, divide the k sub-data blocks in the i-th sub-strip into i′ equal groups;

[0082] When k is not divisible by i′, divide the k sub-data blocks in the i-th sub-strip into i′ groups;

[0083] Among them, the first group to the (i′ - 1)-th group include sub-data blocks, the i′th group includes sub-data blocks.

[0084] In a specific implementation, when the k sub-data blocks in the i-th sub-strip are divided into i′ groups, there are the following two cases:

[0085] 1. If k is divisible by i′, the k sub-data blocks in the i-th sub-strip can be evenly divided into i′ groups, and the first group to the i′-1 group include sub-data blocks, the i′th group includes Sub-data blocks, Representatives Round up. Since k is divisible by i′, mod(k,i′)=0, that is, the number of sub-data blocks included in the i′th group is the same as the number of sub-data blocks included in the first group to the i′-1th group.

[0086] 2. If k is not divisible by i′, the k sub-data blocks in the ith sub-strip are still divided into i′ groups. Since k is not divisible by i′, mod(k,i′)≠0, that is, the k sub-data blocks in the ith sub-strip cannot be evenly divided into i′ groups. The number of sub-data blocks in the first group to the i′-1 group is the same, and they are all The i′th group includes sub-data blocks, that is, all the redundant sub-data blocks are placed in the i′th group.

[0087] Furthermore, in some embodiments, in step 130, calculating the check information according to the grouping and embedding the check information into the sub-check blocks in the corresponding sub-strip may include:

[0088] For the jth packet in the i-th substrip, 1≤j≤i′, 0 <i≤t:

[0089] Obtain (t-i+1) linearly independent check information, and sequentially embed the (t-i+1) linearly independent check information into the γth sub-check block in the jth sub-strip, where i≤γ≤m;

[0090] For the i″th sub-strip, 1 <i″≤t:

[0091] Obtain (i″-1) linearly independent check information, and embed the (i″-1) linearly independent check information into the γ′-th sub-check block in the i″-th sub-strip, 1≤γ′≤i″-1;

[0092] For the i′′′th sub-strip, 1≤i′′ <t:

[0093] Obtain t linearly independent parity information;

[0094] Respectively take the negative values of the t linearly independent parity information, and embed the t linearly independent parity information after taking the negative values into m sub - parity blocks in the (i″′ + 1)-th sub - stripe.

[0095] In a specific implementation, for the sub - data block in the j - th group of the i - th sub - stripe, where 1 ≤ j ≤ i′, 0 < i ≤ t, calculate (t - i + 1) linearly independent parity information based on RS coding, and embed these parity information into the γ - th parity sub - block of the j - th sub - stripe in turn (1 ≤ γ ≤ m).

[0096] For the i″ - th sub - stripe, 1 < i″ ≤ t, calculate (i″ - 1) linearly independent parity information about this sub - stripe based on RS coding, and embed these parity information into the γ′ - th sub - parity block in the i″ - th sub - stripe, where 1 ≤ γ′ ≤ i″ - 1.

[0097] For the i″′ - th sub - stripe, 1 ≤ i″′ < t, calculate t linearly independent parity information based on RS coding, respectively take the negative values of these parity information, and embed the t linearly independent parity information after taking the negative values into m sub - parity blocks in the (i″′ + 1)-th sub - stripe.

[0098] Exemplarily, refer to Figure 2 , in the embodiment of the present application, the erasure - code parameter is (n,k)=(9,6).

[0099] As Figure 2 shown in (1) of [], taking the RS code as the basic code, divide an original data of size 6M into 6 data blocks O1,...,O6, and through RS coding, generate 3 linearly independent parity blocks, assumed to be f(O), g(O), h(O) respectively. These 6 data blocks and 3 parity blocks form a stripe, where the size of each data block and parity block is M′.

[0100] As Figure 2 shown in (2) of [], divide a stripe generated by RS coding into 3 sub - stripes a, b, c. Each sub - stripe contains 6 sub - data blocks and 3 sub - parity blocks, and the size of each sub - data block and sub - parity block is M′ / 3.

[0101] As Figure 2As shown in (3), the sub-data blocks in each sub-strip are grouped in sequence. Suppose the first sub-strip a is divided into 1 group, with 6 data blocks in each group, denoted as {a1, a2,..., a6}; the second strip b is divided into 2 groups, with 3 sub-data blocks in each group, namely {b1, b2, b3} and {b4, b5, b6}; the third sub-strip c is divided into 3 groups, with 2 sub-data blocks in each group, namely {c1, c2}, {c3, c4}, and {c5, c6}.

[0102] As Figure 2 shown in (4), for the sub-data block in the j-th grouping of the i-th sub-strip, where 1 ≤ j ≤ i′, calculate (t - i + 1) linearly independent parity-check messages, and embed these parity-check messages into the γ-th parity-check sub-block of the j-th sub-strip in sequence, where 1 ≤ γ ≤ m.

[0103] Specifically, calculate 3 linearly independent parity-check messages for the first sub-strip a, denoted as f(a), g(a), and h(a) based on RS coding, and embed them into the 3 sub-parity-check blocks of the first sub-strip a respectively; calculate 2 linearly independent parity-check messages for the sub-data blocks in the first grouping of the second sub-strip b (denoted as {b1, b2, b3}), denoted as g(b 123 ), and h(b 123 ), and embed them into the last two sub-parity-check blocks of the first sub-strip a respectively. For example, embed g(b 123 ) into the second sub-parity-check block of the first sub-strip a and embed h(b 123 ) into the third sub-parity-check block of the first sub-strip a. Similarly, embed the 2 linearly independent parity-check messages (denoted as g(b 456 ) and h(b 456 )) for the sub-data blocks in the second grouping of the second sub-strip b into the last 2 sub-parity-check blocks of the second sub-strip b respectively; calculate the parity-check messages for the sub-data blocks in the 3 groupings of the third sub-strip c respectively (denoted as {h(c 12 ), h(c 34 ), h(c 56 )}), and embed them into the last 1 sub-parity-check block of the third sub-strip c.

[0104] As Figure 2 shown in (5), for the i″-th sub-strip, where 1 < i″ ≤ t, calculate (i″ - 1) linearly independent parity-check messages regarding this sub-strip, and embed them into the γ′-th sub-parity-check block in the i″-th sub-strip, where 1 ≤ γ′ ≤ i″ - 1.

[0105] Specifically, calculate 1 parity information (assumed to be f(b)) of the second sub-strip b and embed it into the first parity block of sub-strip b; calculate 2 linearly independent parity information (f(c), g(c)) of the third sub-strip c and embed them into the first 2 sub-parity blocks of sub-strip c respectively.

[0106] As Figure 2 shown in (6) therein, for the i″′-th sub-strip, 1 ≤ i″′ < t, calculate t linearly independent parity information based on RS coding, take the negative of each of the t linearly independent parity information, and embed the t linearly independent parity information after taking the negative into m sub-parity blocks in the (i″′ + 1)-th sub-strip.

[0107] Specifically, calculate 3 linearly independent parity information (assumed to be f(a), g(a), h(a)) of the first sub-strip a, take the negative to get -f(a), -g(a), -h(a), and embed them into 3 sub-parity blocks of the second sub-strip b; similarly, calculate 3 linearly independent parity information (assumed to be f(b), g(b), h(b)) of the second sub-strip b, take the negative to get -f(b), -g(b), -h(b), and embed them into 3 sub-parity blocks of the third sub-strip c respectively.

[0108] The coding scheme generated in this way can effectively reduce the repair bandwidth when a single storage node fails in a distributed storage system using erasure codes.

[0109] Taking the failure of the information (assumed to be {a1, b1, c1}) in the first storage node as an example. Read the data {a2,..., a6, f(a)} from other storage nodes. According to the MDS property of the underlying RS coding, a1 can be recovered. Based on the existing information {a1, a2,..., a6}, calculate g(a); read the second sub-parity block g(a) + g(b 123 ) of sub-strip a and cancel the known g(a) to get g(b 123 ); read {b2, b3} and based on g(b 123 ), b1 can be recovered. Read the third sub-parity block h(a) + h(b 123 ) + h(c 12 ) of sub-strip a, calculate h(a) and h(b 123 ) based on the known information, cancel them from this sub-parity block to get h(c 12 ) ), then read c2 and recover c1, thus recovering all the failed information in node 1. In this process, a total of 11 sub-blocks are read and transmitted in the network. Compared with the underlying RS coding which needs to read and transmit 18 sub-blocks, the repair bandwidth is reduced by 39%.

[0110] An embodiment of the present application provides a method for encoding erasure codes with low repair bandwidth. When repairing failed blocks of erasure codes, the method can directly obtain the failed data block information through the embedded verification information of a specific sub-data block group to complete the repair operation, thereby avoiding reading too much sub-block information from other nodes and reducing the repair bandwidth of the distributed storage system.

[0111] The following describes a low-repair-bandwidth erasure code encoding device provided by the present invention. The low-repair-bandwidth erasure code encoding device described below and the low-repair-bandwidth erasure code encoding method described above can refer to each other.

[0112] See also Figure 3 , an embodiment of the present application provides an erasure code encoding device with low repair bandwidth, which may include: an acquisition module 310, a grouping module 320 and an encoding module 330.

[0113] An acquisition module 310 is configured to divide a stripe including k data blocks and m check blocks into t sub-stripes, each sub-strip including k sub-data blocks and m sub-check blocks, where t=m;

[0114] A grouping module 320 is configured to group the k sub-data blocks in each sub-strip;

[0115] The encoding module 330 is configured to calculate check information according to the grouping, and embed the check information into the sub-check blocks in the corresponding sub-strip.

[0116] An embodiment of the present application provides an erasure code encoding device with low repair bandwidth. By forming a stripe with k data blocks and m check blocks, in a distributed storage system using an erasure code encoding method, the stripe is sub-striped and the check information of the sub-data blocks in some sub-stripes is embedded into the sub-check blocks of other sub-stripes. Therefore, when a storage node in a distributed storage system fails, the repair bandwidth of some sub-stripes can be reduced by using the embedded check information.

[0117] It is understandable that the detailed functional implementation of each of the above units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0118] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method and will not be repeated here.

[0119] Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, see Figure 4The electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call logic instructions in the memory 430 to execute the method in the above embodiment.

[0120] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0121] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0122] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0123] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0124] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0125] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0126] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

[0127] It is easy for those skilled in the art to understand that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for encoding erasure codes with low repair bandwidth, characterized in that: include: Will include data blocks and The stripes of parity blocks are divided into sub-strips, each sub-strip includes sub-data blocks and Sub-check blocks, ; For each sub-strip Sub-data blocks are grouped; Calculating check information according to the grouping, and embedding the check information into the sub-check blocks in the corresponding sub-strip; For each sub-strip The sub-data blocks are grouped, including: exist Can be In the case of divisibility, In the sub-strip The sub-data blocks are divided into groups; exist Cannot be In the case of divisibility, In the sub-strip Sub-data blocks are divided into groups; Among them, the first group to the The groups include Sub-data blocks, The groups include sub-data blocks; The calculating the check information according to the grouping and embedding the check information into the sub-check blocks in the corresponding sub-strip includes: For the The first groups, , : Get Linearly independent check information, and the Linearly independent check information is embedded into the first The first In the sub-check blocks, ; For the Sub-strips, : Get Linearly independent check information, and the Linearly independent check information is embedded into the The first In the sub-check blocks, ; For the Sub-strips, : Get Linearly independent check information; Respectively The linearly independent check information is negative, and the negative Linearly independent check information is embedded into the In the sub-strip The sub-check blocks.

2. The low repair bandwidth erasure coding method according to claim 1, wherein: The will include data blocks and The stripes of parity blocks are divided into sub-strips, including: Divide each data block into sub-data blocks; Divide each check block into Sub-check blocks; Any sub-stripe is determined according to any sub-data block in each data block and any sub-check block in each check block.

3. The low repair bandwidth erasure coding method according to claim 1, wherein: The The sub-data blocks are grouped, including: for The first Sub-strips, : The first In the sub-strip Sub-data blocks are divided into groups, .

4. The low repair bandwidth erasure coding method according to any one of claims 1 to 3, wherein: Said inclusion data blocks and The stripes of parity blocks are obtained as follows: Divide the original data into data blocks; Based on the erasure code encoding method, Data blocks are encoded to generate check blocks; According to the data blocks and the A check block is obtained to obtain the stripe.

5. A low-repair-bandwidth erasure code encoding device, characterized in that: include: Get the module that contains data blocks and The stripes of parity blocks are divided into sub-strips, each sub-strip includes sub-data blocks and Sub-check blocks, ; The grouping module is used to group the Sub-data blocks are grouped; an encoding module, configured to calculate check information according to the grouping, and embed the check information into a sub-check block in the corresponding sub-strip; For each sub-strip The sub-data blocks are grouped, including: exist Can be In the case of divisibility, In the sub-strip The sub-data blocks are divided into groups; exist Cannot be In the case of divisibility, In the sub-strip Sub-data blocks are divided into groups; Among them, the first group to the The groups include Sub-data blocks, The groups include sub-data blocks; The calculating the check information according to the grouping and embedding the check information into the sub-check blocks in the corresponding sub-strip includes: For the The first groups, , : Get Linearly independent check information, and the Linearly independent check information is embedded into the first The first In the sub-check blocks, ; For the Sub-strips, : Get Linearly independent check information, and the Linearly independent check information is embedded into the The first In the sub-check blocks, ; For the Sub-strips, : Get Linearly independent check information; Respectively The linearly independent check information is negative, and the negative Linearly independent check information is embedded into the In the sub-strip The sub-check blocks.

6. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the low repair bandwidth erasure coding method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the erasure coding method with low repair bandwidth according to any one of claims 1 to 4.

8. A computer program product, characterized in that When the computer program product runs on a processor, the processor is enabled to execute the erasure coding method with low repair bandwidth according to any one of claims 1 to 4.

Images