Method and apparatus for generating check blocks

Abstract

The one or more embodiments of the specification provide a generation method and device of a check block. The method comprises: determining a global encoding matrix of a plurality of original data blocks for which a check block is to be generated; determining, for each local domain obtained by dividing the plurality of original data blocks, a local encoding matrix of the local domain based on the global encoding matrix, each local domain including original data blocks belonging to the local domain; generating a global check block of the plurality of original data blocks based on the global encoding matrix, and generating a local check block of each local domain based on the local encoding matrix of each local domain, so that the global check block and the local check blocks of the local domains have a derivation relationship.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of data storage, and in particular, to a method and apparatus for generating a parity block. BACKGROUND

[0002] With the development of technology and the increasing demand, the scale of data storage is expanding, and the existing data storage scheme is mostly implemented in combination with a distributed system with high scalability. In the distributed system, data is stored in the form of data blocks, and a file is decomposed into multiple data blocks stored on different disks or storage nodes.

[0003] In order to ensure the recoverability of data, a parity block (Parity Block) is generally set for the original data block in the distributed system, and the parity block can be divided into a global parity block (Global Parity Block) and a local reconstruction parity block (Local Reconstruction Parity Block). In the case of partial loss of original data blocks, the lost original data blocks can be recovered based on the non-lost original data blocks and the parity block, and in the case of loss of the parity block, the lost parity block can also be reproduced based on the original data block. In related technologies, in order to recover the lost parity block, a large amount of traffic and time is consumed in obtaining the original data block, which reduces the efficiency of data recovery. SUMMARY

[0004] Therefore, one or more embodiments of the present specification provide a method and apparatus for generating a parity block.

[0005] To achieve the above object, one or more embodiments of the present specification provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of the present specification, a method for generating a parity block is provided, the method comprising:

[0007] For a plurality of original data blocks to be generated, a global encoding matrix of the plurality of original data blocks is determined;

[0008] For each local domain obtained by dividing the plurality of original data blocks, a local encoding matrix of the local domain is determined based on the global encoding matrix; each local domain includes original data blocks belonging to the local domain;

[0009] A global parity block of the plurality of original data blocks is generated based on the global encoding matrix, and a local parity block of each local domain is generated based on the local encoding matrix of each local domain, so that the global parity block and the local parity block of each local domain have a derivation relationship.

[0010] According to a second aspect of one or more embodiments of the present specification, a generation apparatus of a check block is provided, the apparatus comprising a global matrix determining unit, a local matrix determining unit and a check block generating unit:

[0011] The global matrix determining unit determines a global encoding matrix of a plurality of original data blocks for which a check block is to be generated;

[0012] The local matrix determining unit determines a local encoding matrix of each local domain obtained by dividing the plurality of original data blocks based on the global encoding matrix; each local domain includes original data blocks belonging to the local domain;

[0013] The check block generating unit generates a global check block of the plurality of original data blocks based on the global encoding matrix, and generates a local check block of each local domain based on the local encoding matrix of each local domain, so that the global check block and the local check block of each local domain have a derivation relationship.

[0014] According to a third aspect of one or more embodiments of the present specification, an electronic device is provided, comprising a processor and a memory for storing processor executable instructions;

[0015] The processor implements the steps in the method of the first aspect described above by running the executable instructions.

[0016] According to a fourth aspect of one or more embodiments of the present specification, a computer readable storage medium is provided, having stored thereon a computer program, the computer program being executed by a processor to implement the steps in the method of the first aspect described above.

[0017] As can be seen from the above description, in the present specification, for a plurality of original data blocks for which a check block is to be generated, a global encoding matrix thereof is first determined, and after the plurality of original data blocks are divided into each local domain, a local encoding matrix of each local domain is determined based on the global encoding matrix, and then a global check block of the plurality of original data blocks is generated based on the global encoding matrix, and a local check block of each local domain is generated based on the local encoding matrix of each local domain, wherein the local encoding matrix used to generate the local check block of each local domain is determined based on the global encoding matrix, so that the local check block and the global check block have a derivation relationship with each other. Based on the derivation relationship, the recovery of the lost check block can be achieved by the non-lost check block, which reduces the traffic and time consumed for obtaining the original data block, optimizes the generation method of the check block, and improves the efficiency of data recovery. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 FIG. 1 is a flowchart of a method for generating a check block according to an example embodiment.

[0019] Figure 2 FIG. 2 is a flowchart of a method for determining a local encoding matrix according to an example embodiment.

[0020] Figure 3 FIG. 3 is a flowchart of a method for generating a global check block and a local check block according to an example embodiment.

[0021] Figure 4 FIG. 4 is a schematic diagram of an electronic device including a device for generating a check block according to an example embodiment.

[0022] Figure 5 FIG. 5 is a block diagram of a device for generating a check block according to an example embodiment. DETAILED DESCRIPTION

[0023] The example embodiments will be described in detail herein with reference to the attached drawings. The following description is with reference to the drawings, in which like numerals represent like elements throughout the several figures. The following description of example embodiments is not representative of all possible embodiments consistent with one or more aspects of the present description. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present description as detailed in the appended claims.

[0024] It should be noted that the steps of the methods in other embodiments need not necessarily be performed in the order shown and described in this specification. In some other embodiments, the steps of the methods can include more or fewer steps than those described in this specification. Furthermore, a single step described in this specification can in other embodiments be split into multiple steps to be described; and multiple steps described in this specification can in other embodiments be combined into a single step to be described.

[0025] With the development of technology and the increasing demand, the scale of data storage is growing, and most of the existing data storage solutions are implemented based on distributed systems with high scalability. In the distributed system, data is stored in the form of data blocks, and all data required to be stored by a file or a storage request will be divided into multiple data blocks and stored on different disks or storage nodes.

[0026] In order to guarantee the recoverability of data, when data is stored in data blocks, a distributed system generally also combines the coding mode of an erasure code (Erasure Code) to set a parity block (Parity Block) for data recovery on the basis of the original data block. The parity block can generally be divided into a global parity block (Global Parity Block) and a local parity block (Local Reconstruction Parity Block), wherein a global parity block of the plurality of original data blocks can be generated based on all the plurality of original data blocks to be generated, and a local parity block of each local domain can be generated for the original data blocks in each local domain after the plurality of original data blocks are respectively divided into different local domains.

[0027] In the case of partial loss of the original data block, the lost original data block can be recovered based on the non-lost original data block and the parity block, and in the case of loss of the parity block, the lost parity block can also be reproduced based on the original data block.

[0028] For example, it is assumed that there are six original data blocks D1, D2, D3, D4, D5, and D6 to be generated, which are respectively divided into two local domains, the local domain F1 includes the original data blocks D1, D2, and D3, and the local domain F2 includes the original data blocks D4, D5, and D6. For the six original data blocks D1, D2, D3, D4, D5, and D6, a global parity block P1 can be generated, P1=D1+2D2+3D3+4D4+5D5+6D6; for the original data blocks D1, D2, and D3 in the local domain F1, a local parity block L1 of the local domain F1 can be generated, L1=D1+D2+D3; and for the original data blocks D4, D5, and D6 in the local domain F2, a local parity block L2 of the local domain F2 can be generated, L2=D4+D5+D6. The "+" operation herein and hereinafter can be implemented by an exclusive or operation.

[0029] In the case of loss of the original data block D1, the lost original data block D1 can be recovered based on the remaining non-lost original data blocks D2, D3 in the local domain F1 to which it belongs and the local parity block L1 of the local domain F1, D1=L1-(D2+D3); or the lost original data block D1 can also be recovered based on other non-lost original data blocks D2, D3, D4, D5, and D6 and the global parity block P1, D1=P1-(2D2+3D3+4D4+5D5+6D6). The "-" operation herein and hereinafter can be implemented based on an exclusive or operation.

[0030] In the case of loss of the global check block P1, the lost global check block P1 can be reproduced based on all the original data blocks D1, D2, D3, D4, D5, D6, P1=D1+D2+D3+D4+D5+D6; in the case of loss of the local check block L1, the lost local check block L1 can be reproduced based on the original data blocks D1, D2, D3 in the local domain F1, L1=D1+D2+D3.

[0031] It can be seen that in the related art, in the case of loss of a check block, the original data blocks need to be acquired to recover the lost check block, the more the number of original data blocks, the greater the flow and time consumption of check block recovery, thereby causing the problem of low data recovery efficiency.

[0032] Therefore, the present specification provides a check block generation method, which determines the local encoding matrix of each local check block based on the global encoding matrix used to generate the global check block, so that the global check block and each local check block have a derivation relationship, thereby reducing the acquisition of original data blocks when recovering the check block, reducing the flow and time consumption, and improving the efficiency of data recovery.

[0033] Please refer to Figure 1 , Figure 1 Fig. 1 shows a flowchart of a check block generation method according to an exemplary embodiment.

[0034] Figure 1 The check block generation method shown in Fig. 1 can be applied to any node device in a distributed system that provides data storage services.

[0035] The check block generation method can include the following specific steps:

[0036] Step 102, for a plurality of original data blocks to be generated, determine the global encoding matrix of the plurality of original data blocks.

[0037] In the distributed system, data is stored in the form of data blocks, and a file to be stored or all data required to be stored by a storage request of a user will be divided into a plurality of original data blocks to be stored. The plurality of original data blocks obtained by division can be stored on different node devices in the distributed system, or can be stored on different disks of the same node device, thereby avoiding the loss of the file or the data required to be stored by the storage request due to the failure of a node device or a disk.

[0038] To realize data recovery, for a plurality of original data blocks to be generated into a check block, any node device in the distributed system can first determine a global encoding matrix of the plurality of original data blocks, to generate a global check block of the plurality of original data blocks based on the global encoding matrix. The any node device can be one of the node devices storing the original data blocks, or a node device in the distributed system for generating a check block.

[0039] The global encoding matrix adopts a preset matrix format, which is determined based on the number of original data blocks to be generated into a check block and the number of global check blocks set in advance. The matrix format of the global encoding matrix has a plurality of selectable implementation formats, and the number of original data blocks to be generated into a check block and the number of global check blocks required to be generated can be set according to actual application scenarios, which is not specifically limited in the present specification.

[0040] For k original data blocks D1, D2,..., D k Generate m global check blocks P1, P2,..., P m For example, k and m are positive integers, and the format of the global encoding matrix is determined by adopting a Vandermonde matrix, and the global encoding matrix is shown in formula (1):

[0041]

[0042] Wherein, the global encoding matrix is composed of m global encoding vectors, and the i-th global encoding vector is composed of k global encoding parameters corresponding to the original data blocks D j One-to-one correspondence. The parameter A ij is a positive integer.

[0043] It should be noted that although the global encoding matrix has a plurality of selectable matrix formats, in order to realize the encoding requirement that the erasure code can still recover the lost original data block when the number of lost original data blocks does not exceed m for k original data blocks and m global check blocks, any global encoding vector in the global encoding matrix should form an invertible matrix with any vector in the unit matrix, and the order of the unit matrix is consistent with the number of original data blocks. The format of the above-mentioned Vandermonde matrix is one of the matrix formats that can meet this encoding requirement.

[0044] Step 104, for each local domain obtained by dividing the plurality of original data blocks, determine a local encoding matrix of the local domain based on the global encoding matrix; each local domain includes original data blocks belonging to the local domain.

[0045] The multiple original data blocks to be generated into the check blocks will be divided into multiple local domains, each local domain including original data blocks belonging to the local domain, and the generation of the local check blocks will be performed for each local domain. The division of the local domains can be implemented in multiple selectable manners, the number of the divided local domains can be pre-set or can vary depending on the number of the original data blocks to be generated into the check blocks, the number of the original data blocks included in each local domain can be consistent or inconsistent, and the original data blocks included in each local domain can be continuous data blocks in the original file or storage request or can be random and discontinuous data blocks. The present specification does not make specific limitations in this regard.

[0046] In one example, the number of the original data blocks included in each local domain can be pre-set, the number of the divided local domains can be determined based on the number of the original data blocks to be generated into the check blocks, and after the number of the local domains and the number of the original data blocks included in each local domain are determined, the multiple original data blocks can be divided into the local domains according to their continuous order in the decomposition. For example, for the original data blocks D1, D2, D3, D4, D5, and D6 to be generated into the check blocks, the number of the original data blocks included in each local domain is pre-set to 3, it is determined that 2 local domains will be divided, and the original data blocks D1, D2, and D3 continuously obtained in the decomposition are divided into the local domain F1, and the original data blocks D4, D5, and D6 are divided into the local domain F2.

[0047] After the division of the local domains is completed, each local domain and the original data blocks included therein can be determined, and for each local domain, a local encoding matrix of the local domain can be determined based on the global encoding matrix determined in step 102, so that the local check block generated based on the local encoding matrix of each local domain and the global check block generated based on the global encoding matrix have a derivation relationship.

[0048] Based on the foregoing, the global encoding matrix includes one or more global encoding vectors, each global encoding vector includes a global encoding parameter corresponding to each original data block, so as to provide k original data blocks D1, D2,..., Dk with m global encoding parameters A1, A2,..., Am. k The m global check blocks P1, P2,..., Pm are generated. m For example, the global encoding matrix includes m global encoding vectors, the ith global encoding vector includes k global encoding parameters A1, A2,..., Am corresponding to the original data blocks D1, D2,..., Dk one by one. j The global encoding parameters A1, A2,..., Am corresponding to the original data blocks D1, D2,..., Dk one by one. ij (i = 1, 2,..., m; j = 1, 2,..., k), and for example, the global encoding matrix shown in formula (1), the ith global encoding vector is shown in formula (2):

[0049]

[0050] Assuming that n local domains are divided, for each local domain F q (q = 1, 2,..., n), the global encoding parameter A ij in the global encoding matrix can be used to determine the local encoding parameter B q in the local encoding matrix of the local domain F qr (r = 1, 2,..., r max , r max is consistent with the number of original data blocks included in the local domain), and the local encoding parameter B qr is used to form the local encoding matrix of the local domain F q , as shown in equation (3):

[0051]

[0052] There are various optional implementation manners for determining the local encoding matrix of the local domain based on the global encoding matrix, which are not specifically limited in the present specification. It can be understood that the logic of determining the local encoding matrix of each local domain based on the global encoding matrix affects the derivation relationship between the global check block and the local check block of each local domain.

[0053] In step 106, the global check block of the plurality of original data blocks is generated based on the global encoding matrix, and the local check block of each local domain is generated based on the local encoding matrix of each local domain, so that the global check block and the local check block of each local domain have a derivation relationship.

[0054] After determining the global encoding matrix and the local encoding matrix of each local domain, the global check block of the plurality of original data blocks can be generated based on the global encoding matrix, the i-th global encoding vector in the global encoding matrix corresponds to the i-th global check block P i ; the local check block of each local domain can be generated based on the local encoding matrix of each local domain, and the local encoding matrix of the q-th local domain F q corresponds to the local check block L q of the local domain F q .

[0055] In one example, based on the global encoding matrix shown in equation (1), m global check blocks P1, P2,..., P m of k original data blocks D1, D2,..., D k are generated, as shown in equation (4):

[0056]

[0057] The global check block

[0058] The global check block can be used to recover the lost original data block. When the number of lost original data blocks in the k original data blocks does not exceed m, assuming that the original data blocks D1 and D2 are lost, the recovery method of the lost original data blocks D1 and D2 is shown in formula (5):

[0059]

[0060] For a local domain F1 including original data blocks D1, D2 and D3, the local encoding matrix of the local domain F1 is shown in formula (6):

[0061] [B 11 B 12 B 13 ] Formula (6)

[0062] Based on the local encoding matrix of the local domain F1 shown in formula (6), the local check block L1 of the local domain F1 is generated, as shown in formula (7):

[0063]

[0064] Wherein, the local check block L1 = B 11 *D1 + B 12 *D2 + B 13 *D3.

[0065] The local check block can be used to recover the lost original data block in the local domain. When the original data block D1 in the local domain F1 is lost, based on the local encoding matrix of the local domain F1 and the non-lost data blocks D2 and D3 in the domain, the lost original data block D1 can be recovered, D1 = (L1-B 12 *D2-B 13 *D3) / B 11 .

[0066] Since the local encoding matrix of each local domain is determined on the basis of the global encoding matrix, the global check block generated based on the global encoding matrix and the local check block of each local domain generated based on the local encoding matrix have a derivation relationship. The derivation relationship between the global check block and the local check block of each local domain in step 106 is determined by the logic of determining the local encoding matrix of each local domain based on the global encoding matrix in step 104.

[0067] As can be seen from the above description, in the present specification, for a plurality of original data blocks to be generated into a check block, a global encoding matrix is first determined, and after the plurality of original data blocks are divided into each local domain, a local encoding matrix of each local domain is determined based on the global encoding matrix, and then a global check block of the plurality of original data blocks is generated based on the global encoding matrix, and a local check block of each local domain is generated based on the local encoding matrix of each local domain. The local encoding matrix used to generate the local check block of each local domain is determined based on the global encoding matrix, so that the local check block and the global check block have a derivation relationship with each other. Based on the derivation relationship, the recovery of the lost check block can be realized by the non-lost check block, the traffic and time consumed for obtaining the original data block are reduced, the generation mode of the check block is optimized, and the efficiency of data recovery is improved.

[0068] In order to enable those skilled in the art to better understand the technical solutions in the present specification, the foregoing content is further described in detail below.

[0069] Please refer to Figure 2 In an alternative implementation, step 104, for each local domain obtained by dividing the plurality of original data blocks, a local encoding matrix of the local domain is determined based on the global encoding matrix; each local domain includes original data blocks belonging to the local domain, which can include the following steps:

[0070] Step 1042, for each original data block in the local domain, based on the global encoding parameters in each global encoding vector of the global encoding matrix corresponding to the original data block, determine the local encoding parameters corresponding to the original data block, and obtain the local encoding parameters corresponding to each original data block in the local domain.

[0071] Step 1044, based on the local encoding parameters corresponding to each original data block in the local domain, determine the local encoding matrix of the local domain.

[0072] Suppose n local domains are divided, for each local domain F q (q = 1, 2,..., n), the local encoding matrix of the local domain F q can be determined. The local encoding matrix includes one-to-one local encoding parameters corresponding to each original data block in the local domain F q , for each original data block D q in the local domain F j , based on the global encoding parameters A j in each global encoding vector of the global encoding matrix corresponding to the original data block D ij , the local encoding parameters Bqj .

[0073] Taking a local field F1 that includes the original parity blocks D1, D2, and D3 as an example, the local coding matrix of the local field F1 includes local coding parameters B that correspond one-to-one with the original data blocks D1, D2, and D3. 11 B 12 B 13 Based on the global coding parameters A corresponding to the original data block D1 in the global coding matrix i1 (i = 1, 2, ..., m) can determine the local coding parameter B. 11 Based on the global coding parameters A corresponding to the original data block D2 in the global coding matrix i2 (i = 1, 2, ..., m) can determine the local coding parameter B. 12 Based on the global coding parameters A corresponding to the original data block D3 in the global coding matrix i3 (i = 1, 2, ..., m) can determine the local coding parameter B. 13 .

[0074] In one example, the local coding parameters corresponding to the original data block can be the sum of its corresponding global coding parameters, then B 11 =A 11 +A 21 +…+A m1 B 12 =A 12 +A 22 +…+A m2 B 13 =A 13 +A 23 +…+A m3 .

[0075] After determining the local coding parameters corresponding to each original data block within the local domain, the local coding matrix of the local domain can be determined based on the local coding parameters. For example, the local coding matrix of the local domain F1 mentioned above is [A 11 +A 21 +…+A m1 A 12 +A 22 +…+A m2 A 13 +A 23 +…+A m3 ].

[0076] It can be understood that the logic of determining the local encoding matrix of each local domain based on the global encoding matrix affects the derivation relationship between the global check block and the local check block of each local domain. Based on the foregoing, when the local encoding parameter corresponding to the original data block in each local domain is the sum of the corresponding global encoding parameter, the value of the global check block generated based on the global encoding matrix in step 106 performs an exclusive or operation, which is consistent with the value of the local check block generated based on the local encoding matrix of each local domain:

[0077]

[0078] Please refer to Figure 3 In an alternative implementation, step 106, generating the global check block of the plurality of original data blocks based on the global encoding matrix, and generating the local check block of each local domain based on the local encoding matrix of each local domain, can include the following steps:

[0079] Step 1062, determining a target encoding matrix based on the global encoding matrix and the local encoding matrix of each local domain.

[0080] After determining the global encoding matrix and the local encoding matrix of each local domain, the target encoding matrix can be determined in combination with the global encoding matrix and the local encoding matrix of each local domain. In one example, the local encoding matrix of each local domain can be zero-padded, and the zero-padded local encoding matrix of each local domain and the global encoding matrix can be spliced to obtain the target encoding matrix.

[0081] The target encoding matrix includes a plurality of target encoding vectors, each target encoding vector corresponding to a check block, and each target encoding vector including a target encoding parameter corresponding to each original data block.

[0082] Wherein, the target encoding vector corresponding to the global check block can adopt the global encoding vector in the global encoding matrix, and the target encoding parameter in the target encoding vector can be the global encoding parameter in the global encoding vector; the target encoding vector corresponding to the local check block of each local domain can be determined based on the local encoding matrix of the local domain, and the target encoding parameter in the target encoding vector corresponding to the original data block in the local domain can be the local encoding parameter in the local encoding matrix, while the target encoding parameter corresponding to the original data block outside the local domain can be 0.

[0083] Step 1064, generating the global check block of the plurality of original data blocks and the local check block of each local domain based on the target encoding matrix.

[0084] generate each global check block based on each global encoding vector in the target encoding matrix, and generate each local check block of each local domain based on the target encoding vector determined by the local encoding matrix of each local domain in the target encoding matrix. It is to be noted that the global check block and the local check block of each local domain can be generated at one time based on the target encoding matrix.

[0085] In this way, by combining the global encoding matrix and the local encoding matrix of each local domain in a target encoding matrix, and generating the global check block and the local check block of each local domain based on the target encoding matrix, compared with generating the global check block and the local check block of each local domain based on the global encoding matrix and the local encoding matrix of each local domain respectively, the operation process is simplified, and the efficiency of generating the check block is improved.

[0086] Alternatively, the method for generating the check block provided in the specification can further include a method for recovering the check block.

[0087] In an alternative implementation, the method further includes a method for recovering the global check block:

[0088] In the case of loss of any global check block, the lost global check block is recovered based on the global check block and the local check block of each local domain which are not lost.

[0089] The way of recovering the global check block depends on the logic used in step 106 to determine the local encoding matrix of each local domain based on the global encoding matrix. As described above, the local encoding parameter corresponding to the original data block in the local domain is the sum of each global encoding parameter corresponding thereto. For example, the deduced relationship between the global check blocks P1, P2,..., P m and the local check blocks L1, L2,..., L n of each local domain is P1+P2+...+P m =L1+L2+...+L n Based on the global check block and the local check block of each local domain which are not lost, any lost global check block P i ,

[0090] In an alternative implementation, the method further includes a method for recovering the local check block:

[0091] In the case of loss of any local check block, the lost local check block is recovered based on the global check block and the local check block of each local domain which are not lost.

[0092] The manner of recovering the global check blocks depends on the logic adopted in step 106 when determining the local encoding matrix of each local domain based on the global encoding matrix. As previously described, the local encoding parameters of the original data blocks in a local domain are the sum of the corresponding global encoding parameters. For example, the derived relationship between the global check blocks P1, P2,..., P m The local check blocks L1, L2,..., L n The derived relationship between the global check blocks P1, P2,..., P m = L1+ L2+... + L n Any lost global check block L q ,

[0093] Compared with the related art, in which all original data blocks need to be obtained when recovering global check blocks and all original data blocks in the corresponding local domain need to be obtained when recovering local check blocks, the method for generating check blocks provided in the present specification can recover any lost global check block or local check block based on the non-lost check blocks, thereby reducing the consumed traffic and time, improving the efficiency of data recovery, and the effect is more significant when the number of original data blocks to be generated is larger.

[0094] In order to enable those skilled in the art to better understand the technical solutions in the present specification, a specific example is used to describe in detail a method for generating check blocks provided in the present specification.

[0095] A node device in a distributed system, for original data blocks D1, D2, D3, D4, D5, D6 to be generated check blocks, and the pre-set number of global check blocks: 2, determines the global encoding matrix of the original data blocks D1, D2, D3, D4, D5, D6 in the format of the Vandermonde matrix as follows:

[0096]

[0097] The original data blocks D1, D2, D3, D4, D5, D6 are divided into two local domains, the local domain F1 includes the original data blocks D1, D2, D3, and the local domain F2 includes the original data blocks D4, D5, D6.

[0098] For the local domain F1, the local encoding matrix is determined, and the global encoding parameters corresponding to the original data block D1 in the global encoding matrix are 1, 1. The local encoding parameters B 11The sum of the two is 2; the corresponding global encoding parameters of the original data block D2 in the global encoding matrix are 1 and 2, and the corresponding local encoding parameters B of D2 are determined 12 3; the corresponding local encoding parameters B of the original data block D3 are determined 13 5, so as to determine the local encoding matrix of the local domain F1 as follows: [2 3 5]

[0100] Similarly, the local encoding matrix of the local domain F2 is determined as follows: [9 17 33]

[0102] Based on the global encoding matrix and the local encoding matrix of each local domain, the target encoding matrix is determined by using zero padding and splicing. The target encoding matrix includes target encoding vectors consistent with the number of check blocks. Each target encoding vector includes target encoding parameters consistent with the number of original data blocks. The target encoding vector corresponding to the global check block uses the global encoding vector, and the target encoding vector corresponding to the local check block is constructed by zero padding based on the local encoding matrix of each local domain. The target encoding matrix obtained by splicing the global encoding matrix and the local encoding matrix after zero padding is as follows:

[0103]

[0104] Based on the target encoding matrix, the global check blocks P1 and P2 of the original data blocks D1, D2, D3, D4, D5, and D6, and the local check blocks L1 of the local domain F1 and the local check blocks L2 of the local domain F2 can be generated as follows:

[0105]

[0106] In the case of loss of any global check block or local check block, based on the derivation relationship P1+P2=L1+L2, the lost global check block or local check block can be recovered based on the non-lost check block.

[0107] As can be seen from the above description, in the present specification, for a plurality of original data blocks to be generated into a check block, a global encoding matrix is first determined, and after the plurality of original data blocks are divided into each local domain, a local encoding matrix of each local domain is determined based on the global encoding matrix, and then the global check block of the plurality of original data blocks is generated based on the global encoding matrix, and the local check block of each local domain is generated based on the local encoding matrix of each local domain. The local encoding matrix used to generate the local check block of each local domain is determined based on the global encoding matrix, so that the local check block and the global check block have a derivation relationship with each other. Based on the derivation relationship, the recovery of the lost check block can be realized by the non-lost check block, the traffic and time consumed for obtaining the original data block are reduced, the generation mode of the check block is optimized, and the efficiency of data recovery is improved.

[0108] Figure 4 is a schematic structural diagram of an electronic device provided by an example embodiment. Please refer to Figure 4 At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410, and of course, other hardware required by the business. One or more embodiments of the present specification can be implemented in a software manner, such as reading the corresponding computer program from the non-volatile memory 410 into the memory 408 by the processor 402 and then running. Of course, in addition to the software implementation, one or more embodiments of the present specification do not exclude other implementation manners, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0109] Please refer to Figure 5 The check block generation device provided by the present specification can be applied to an electronic device as shown in Figure 4 The check block generation device can include a global matrix determination unit 510, a local matrix determination unit 520, and a check block generation unit 530:

[0110] The global matrix determination unit 510 determines the global encoding matrix of the plurality of original data blocks to be generated into a check block;

[0111] The local matrix determination unit 520 determines the local encoding matrix of each local domain based on the global encoding matrix for each local domain obtained by dividing the plurality of original data blocks; each local domain includes original data blocks belonging to the local domain;

[0112] The check block generation unit 530 generates a global check block of the plurality of original data blocks based on the global encoding matrix, and generates a local check block of each local domain based on the local encoding matrix of each local domain, so that the global check block and the local check block of each local domain have a derivation relationship.

[0113] Optionally, the global encoding matrix comprises one or more global encoding vectors, and each global encoding vector comprises a global encoding parameter corresponding to each original data block.

[0114] The local matrix determination unit 520, when determining the local encoding matrix of each local domain obtained by dividing the plurality of original data blocks based on the global encoding matrix, comprises:

[0115] For each original data block in the local domain, based on the global encoding parameter corresponding to the original data block in each global encoding vector, the local encoding parameter corresponding to the original data block is determined, and the local encoding parameter corresponding to each original data block in the local domain is obtained.

[0116] Based on the local encoding parameter corresponding to each original data block in the local domain, the local encoding matrix of the local domain is determined.

[0117] Optionally, the check block generation unit 530, when generating the global check block of the plurality of original data blocks based on the global encoding matrix, and generating the local check block of each local domain based on the local encoding matrix of each local domain, comprises:

[0118] Determine the target encoding matrix based on the global encoding matrix and the local encoding matrix of each local domain.

[0119] Generate the global check block of the plurality of original data blocks and the local check block of each local domain based on the target encoding matrix.

[0120] Optionally, the device further comprises:

[0121] The check block recovery unit 540 recovers the lost global check block based on the non-lost global check block and the local check block of each local domain in the case of any global check block loss.

[0122] Optionally, the device further comprises:

[0123] The check block recovery unit 540 recovers the lost local check block based on the global check block and the non-lost local check block of each local domain in the case of any local check block loss.

[0124] The systems, apparatuses, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an e-mail device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0125] In one typical configuration, a computer includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0126] The memory can include non-persistent memory, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or Flash memory, among others. The memory is an example of computer-readable media.

[0127] Computer-readable media includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology for storing information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media, or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to computing devices. According to the definition herein, computer-readable media does not include transitory media, such as modulated data signals and carriers.

[0128] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article, or device including the element.

[0129] The above-described embodiments of the application have several aspects, no single one of which is solely responsible for the application's desirable attributes. Without limiting the scope of this application as expressed by the claims which follow, some further examples of aspects of the application are discussed below.

[0130] The terminology used in this description is for the purpose of describing particular embodiments only and is not intended to limit one or more embodiments of the present description. As used in one or more embodiments of the present description and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0131] It is to be understood that the terms first, second, third, etc. can be used herein to describe various information, but such information should not be limited by these terms. These terms are used only to distinguish one piece of information from another. For example, a first information can also be termed a second information, and, similarly, a second information can also be termed a first information, without departing from the scope of one or more embodiments of the present description. The word "if' as used herein means "when" or "upon" or "in response to the determination" depending on the context.

[0132] The foregoing is considered as illustrative only of the principles of one or more embodiments of the present description. Further, since numerous modifications and changes can be made to the subject matter of one or more embodiments of the present description without departing from the scope of one or more embodiments of the present description, it is our intention that all such modifications and changes be considered as within the scope of one or more embodiments of the present description. Accordingly, the application is to be realized in a manner that obviates the aforementioned deficiencies.

Claims

1. A method for generating a check block, the method comprising: For multiple raw data blocks to be used to generate a verification block, determine the global encoding matrix of the multiple raw data blocks; For each local domain obtained by dividing the multiple original data blocks, a local encoding matrix of the local domain is determined based on the global encoding matrix; each local domain includes the original data block belonging to the local domain; A global check block is generated based on the global encoding matrix of the multiple original data blocks, and a local check block is generated based on the local encoding matrix of each local domain, so that there is a derivation relationship between the global check block and the local check block of each local domain. The derivation relationship is used to jointly realize the recovery of lost check blocks by combining the unlost check blocks. Also includes: In the event of the loss of any global check block, the lost global check block is recovered based on the unlost global check blocks and the local check blocks of each local domain. In the event of the loss of any local check block, the lost local check block is recovered based on the global check block and the local check blocks of each local domain that are not lost.

2. The method according to claim 1, wherein the global encoding matrix includes one or more global encoding vectors, and each global encoding vector includes global encoding parameters corresponding one-to-one with each original data block; For each local region obtained by dividing the multiple original data blocks, the local coding matrix of the local region is determined based on the global coding matrix, including: For each original data block within the local domain, based on the global coding parameters corresponding to the original data block in each global coding vector, the local coding parameters corresponding to the original data block are determined, thus obtaining the local coding parameters corresponding to each original data block within the local domain. Based on the local coding parameters corresponding to each original data block within the local domain, the local coding matrix of the local domain is determined.

3. The method according to claim 1, wherein generating a global check block for the plurality of original data blocks based on the global encoding matrix, and generating a local check block for each local domain based on the local encoding matrix of each local domain, comprises: The target encoding matrix is ​​determined based on the global encoding matrix and the local encoding matrices of each local domain; Based on the target encoding matrix, a global check block and local check blocks for each local domain of the plurality of original data blocks are generated.

4. A parity block generation apparatus, the apparatus comprising a global matrix determination unit, a local matrix determination unit, and a parity block generation unit: The global matrix determination unit determines the global encoding matrix of the multiple original data blocks for which a verification block is to be generated; The local matrix determination unit determines the local encoding matrix of each local domain obtained by dividing the multiple original data blocks, based on the global encoding matrix; each local domain includes the original data block belonging to the local domain. The check block generation unit generates a global check block for the plurality of original data blocks based on the global encoding matrix, and generates local check blocks for each local domain based on the local encoding matrix of each local domain, so that there is a derivation relationship between the global check block and the local check blocks of each local domain. The derivation relationship is used to jointly realize the recovery of lost check blocks by combining the unlost check blocks. The check block recovery unit recovers the lost global check block based on the unlost global check block and the local check blocks of each local domain when any global check block is lost; and recovers the lost local check block based on the global check block and the unlost local check blocks of each local domain when any local check block is lost.

5. The apparatus according to claim 4, wherein the global encoding matrix includes one or more global encoding vectors, and each global encoding vector includes global encoding parameters corresponding one-to-one with each original data block; The local matrix determination unit, when determining the local encoding matrix of each local domain obtained by dividing the multiple original data blocks, based on the global encoding matrix, includes: For each original data block within the local domain, based on the global coding parameters corresponding to the original data block in each global coding vector, the local coding parameters corresponding to the original data block are determined, thus obtaining the local coding parameters corresponding to each original data block within the local domain. Based on the local coding parameters corresponding to each original data block within the local domain, the local coding matrix of the local domain is determined.

6. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor implements the steps of the method as described in any one of claims 1-3 by running the executable instructions.

7. A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the method as claimed in any one of claims 1-3.

Images