Ec decoding method and system for distributed clusters

Abstract

The application discloses an EC decoding method and system of a distributed cluster, relates to the technical field of cloud computing, and can be applied to the technical field of cloud computing, big data, intelligent supply chain and the like. The distributed cluster uses an error correction code (EC) coding mode to determine data object blocks for storage, and the data object blocks comprise data blocks and check blocks. One specific implementation of the method comprises the following steps: in response to an exception of a data block of any data object stored by a target node, obtaining the identities of a plurality of decoding nodes storing the data object blocks; sending a decoding request to the decoding nodes according to the identities, so that the decoding nodes perform calculation according to the locally stored data object blocks; and obtaining an abnormal data block recovered based on the final calculation result of each decoding node. The implementation can reduce the network overhead and CPU resource occupation of a single node in the data block recovery process.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to an EC decoding method and system for a distributed cluster. Background Art

[0002] In distributed computer clusters, to ensure that the system can continue to operate normally even after certain data corruption, EC (Erasure Code) technology is often used for data storage. This technology primarily uses the EC algorithm to encode the original data blocks into data blocks and check blocks, which are then stored together for fault tolerance. If a data block is abnormal, the abnormal data block can be recovered based on other blocks and the EC decoding algorithm. In current distributed clusters, if a node's data block is abnormal, the node must pull the data block or check block from other nodes and execute the EC decoding algorithm locally, which results in high network overhead and CPU resource utilization for the node. Summary of the Invention

[0003] In view of this, an embodiment of the present invention provides a distributed cluster EC decoding method and system, which can reduce the network overhead and CPU resource usage of a single node during the data block recovery process.

[0004] To achieve the above object, according to one aspect of the present invention, a distributed cluster EC decoding method is provided.

[0005] For the EC decoding method of a distributed cluster in an embodiment of the present invention, the distributed cluster uses an erasure code (EC) encoding method to determine the blocks of a data object for storage, and the blocks include: data blocks and check blocks; the method is executed by a target node in the distributed cluster, and the method includes: in response to an abnormality in a data block of any data object stored by the target node, obtaining the identifiers of multiple decoding nodes that store the blocks of the data object; sending a decoding request to the decoding node according to the identifier, so that the decoding node performs calculations based on the blocks of the data object stored locally; and obtaining the abnormal data blocks recovered based on the final calculation results of each decoding node.

[0006] Optionally, the blocks of the data object stored by each decoding node are in a normal state, and each node storing the blocks of the data object pre-stores: the identifiers of other nodes storing the blocks of the data object; and obtaining the identifiers of multiple decoding nodes storing the blocks of the data object includes: communicating with other nodes storing the blocks of the data object in a preset order to detect whether the blocks stored by the communicating nodes are normal, thereby determining the identifiers of the multiple decoding nodes; wherein the order includes ascending or descending order of the node identifiers, or ascending order of the network distance from the target node.

[0007] Optionally, sending a decoding request to the decoding node according to the identifier includes: sending a decoding request to one of the multiple decoding nodes; wherein the decoding request is transmitted in a point-to-point manner among the multiple decoding nodes until each decoding node receives the decoding request.

[0008] Optionally, the multiple decoding nodes form a chain structure, and the number of the multiple decoding nodes is equal to the total number of data blocks formed by the data object being divided; when the total number of data blocks is equal to 2, the multiple decoding nodes include a head node and a tail node; when the total number of data blocks is greater than 2, the multiple decoding nodes include a head node, an intermediate node and a tail node; the final calculation result of the tail node is the local calculation result of the tail node, the final calculation result of any intermediate node is the sum of the local calculation result of the intermediate node and the final calculation result of the downstream node of the intermediate node, and the final calculation result of the head node is the recovered abnormal data block.

[0009] Optionally, the decoding request indicates: the identifier of the data object, the identifier of the block of the data object stored by each decoding node, and the identifier of the abnormal data block stored by the target node; and the local calculation result of any decoding node is the product of the block stored by the decoding node and the target element, the target element is determined by the decoding node from the row vector based on the arrangement position of the locally stored block in the blocks stored by the multiple decoding nodes, the row vector is the row vector corresponding to the abnormal data block determined by any decoding node from the EC decoding matrix after receiving the decoding request, and the EC decoding matrix is ​​obtained by any decoding node based on the pre-stored EC encoding matrix used by the data object and the identifier of the block of the data object stored by each decoding node indicated by the decoding request.

[0010] Optionally, the acquiring of the abnormal data block recovered based on the final calculation result of each decoding node includes: acquiring the recovered abnormal data block sent by the head node.

[0011] Optionally, the decoding request further indicates the identifier of a decoding node that has not currently received the decoding request; when the decoding request involves multiple data objects, the information of the multiple data objects is arranged in the decoding request according to the disk storage order of the multiple data objects on the target node.

[0012] To achieve the above object, according to another aspect of the present invention, a distributed cluster EC decoding method is provided.

[0013] For the EC decoding method of a distributed cluster in an embodiment of the present invention, the distributed cluster uses an erasure code (EC) encoding method to determine the blocks of a data object for storage, and the blocks include: data blocks and check blocks; the method is executed by multiple decoding nodes in the distributed cluster that store blocks of the same data object, and the method includes: when an abnormality occurs in the data block of the data object stored in the target node of the distributed cluster, receiving a decoding request sent by the target node; performing calculations based on the decoding request and the blocks of the data object stored locally by the decoding node, and obtaining recovered abnormal data blocks based on the final calculation results of the decoding node; and returning the recovered abnormal data blocks to the target node.

[0014] Optionally, the blocks of the data object stored by each decoding node are in a normal state; and the receiving of the decoding request sent by the target node includes: based on one decoding node receiving the decoding request sent by the target node, transmitting the decoding request in a point-to-point manner within the multiple decoding nodes until each decoding node receives the decoding request.

[0015] Optionally, the multiple decoding nodes form a chain structure, and the number of the multiple decoding nodes is equal to the total number of data blocks formed by dividing the data object; when the total number of data blocks is equal to 2, the multiple decoding nodes include a head node and a tail node; when the total number of data blocks is greater than 2, the multiple decoding nodes include a head node, an intermediate node and a tail node.

[0016] Optionally, the calculation based on the decoding request and the blocks of the data object stored locally at the decoding node includes: for the tail node, returning the local calculation result of the tail node as the final calculation result of the tail node to the upstream node of the tail node; for any intermediate node, returning the sum of the local calculation result of the intermediate node and the final calculation result of the downstream node of the intermediate node as the final calculation result of the intermediate node to the upstream node of the intermediate node; for the first node, adding the local calculation result of the first node and the final calculation result of the downstream node of the first node to obtain the recovered abnormal data block.

[0017] Optionally, the decoding request indicates: the identifier of the data object, the identifier of the block of the data object stored by each decoding node, and the identifier of the abnormal data block stored by the target node; and any decoding node determines the local calculation result through the following steps: in response to receiving the decoding request, determining the EC decoding matrix based on the pre-stored EC encoding matrix used by the data object and the identifier of the block of the data object stored by each decoding node indicated by the decoding request; determining the row vector corresponding to the abnormal data block from the EC decoding matrix, and determining the target element from the row vector based on the arrangement position of the block stored by any decoding node in the blocks stored by the multiple decoding nodes; multiplying the block stored by any decoding node by the target element to obtain the local calculation result of any decoding node.

[0018] Optionally, returning the recovered abnormal data block to the target node includes: returning the recovered abnormal data block to the target node based on the first node.

[0019] Optionally, each node storing the blocks of the data object pre-stores: the identifiers of other nodes storing the blocks of the data object, and the decoding request further indicates the identifiers of decoding nodes that have not currently received the decoding request; in the case where the decoding request involves multiple data objects, the information of the multiple data objects is arranged in the decoding request according to the disk storage order of the multiple data objects on the target node.

[0020] To achieve the above object, according to another aspect of the present invention, a distributed cluster EC decoding system is provided.

[0021] For the EC decoding system of a distributed cluster in an embodiment of the present invention, the distributed cluster uses an erasure code (EC) encoding method to determine the blocks of a data object for storage, and the blocks include: data blocks and check blocks. The system includes a target node and a decoding node that store the blocks of the same data object; wherein, when the target node determines that an abnormality occurs in a data block of the data object stored locally, the target node obtains the identifiers of multiple decoding nodes and sends a decoding request to the decoding node based on the identifiers; after receiving the decoding request, the decoding node performs calculations based on the decoding request and the blocks of the data object stored locally at the decoding node, obtains a recovered abnormal data block based on the final calculation result of the decoding node, and returns the recovered abnormal data block to the target node.

[0022] To achieve the above objective, according to another aspect of the present invention, an electronic device is provided.

[0023] An electronic device of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the distributed cluster EC decoding method provided by the present invention.

[0024] To achieve the above objective, according to another aspect of the present invention, a computer-readable storage medium is provided.

[0025] A computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements the EC decoding method of a distributed cluster provided by the present invention.

[0026] According to the technical solution of the present invention, the embodiments of the above invention have the following advantages or beneficial effects:

[0027] In a distributed cluster using erasure codes (EC) for data storage, if a target node stores an abnormal data block, the target node first tentatively determines the identities of multiple decoding nodes storing the corresponding data object's partitions through active discovery. The partitions stored by these decoding nodes are in a normal state and can be used to recover the abnormal data block. The number of decoding nodes is equal to the total number of data blocks formed by partitioning the data object, thus satisfying the EC decoding execution conditions. The target node then sends a decoding request to a decoding node. After receiving the decoding request, each decoding node sends a decoding request to a decoding node that has not yet received the decoding request, and this continues until every decoding node receives the decoding request. That is, after the target node sends a decoding request to a decoding node, each decoding node sends a chain of decoding requests in a point-to-point communication until every decoding node receives the decoding request, and each decoding node only receives the decoding request once. After receiving the decoding request, each decoding node performs calculations based on the EC decoding matrix and the locally stored partitions. The target node ultimately obtains the recovered abnormal data block based on the final calculation results of each decoding node. In this way, the network resources required for data transmission and the computing resources required for EC decoding are distributed between the target node and each decoding node, avoiding the high network overhead and CPU usage of a single node, while improving EC decoding efficiency and reducing data block recovery time.

[0028] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0030] Figure 1 It is a schematic diagram of the known EC algorithm principle;

[0031] Figure 2 It is a schematic diagram of EC decoding of a distributed cluster in the prior art;

[0032] Figure 3 Schematic diagram of the principle of the distributed cluster EC decoding system in an embodiment of the present invention;

[0033] Figure 4 1 is a schematic diagram of the main steps of the EC decoding method of the distributed cluster executed by the target node in an embodiment of the present invention;

[0034] Figure 5 1 is a schematic diagram of the main steps of the distributed cluster EC decoding method executed by the decoding node in an embodiment of the present invention;

[0035] Figure 6 3 is a schematic diagram of the structure of an electronic device used to implement the EC decoding method of a distributed cluster in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0037] The following first explains the principle of the EC algorithm, see Figure 1 .

[0038] When storing a data object in a distributed cluster, the data object is first divided into multiple data blocks, such as Figure 1 The data object can be a BLOB (Binary Large OBject), a file, or a data block formed by segmentation. Then, the encoding matrix is ​​multiplied by the column matrix formed by arranging the data blocks in a preset order (for example, from D1 to D5), and the data blocks and at least one check block ( Figure 1 The first n rows (n is the total number of data blocks formed by segmentation, an integer greater than 1) of the encoding matrix are the n-order unit matrix, and the last m rows (such as Figure 1Each element in the high grayscale position in the encoding matrix (m is the total number of check blocks, which is an integer not less than 1) is a non-zero real number. It can be seen that each row of the encoding matrix corresponds to a block (data block or check block). In addition, the encoding matrix needs to satisfy the requirement that the matrix consisting of any n rows is reversible to perform subsequent decoding calculations. The above data blocks and check blocks are blocks of data objects and are stored in different nodes of the distributed cluster. In some scenarios, the same node can store at most one block of the same data object.

[0039] The above mechanism can tolerate at most m blocks that are abnormal (such as lost, deleted, or have errors, etc.), that is, at least n blocks need to be in a normal state (that is, the block data is not lost and can be read) to recover the abnormal block. Figure 1 In the process, when D1, D4, and C2 are abnormal, the rows corresponding to these three blocks in the encoding matrix are deleted to obtain a square matrix. After that, the inverse matrix is ​​calculated to obtain the decoding matrix. Finally, the decoding matrix is ​​multiplied by the column matrix composed of the current blocks (the arrangement order of the blocks is the same as the arrangement order of the encoding process, except that the abnormal blocks are removed) to restore all the data blocks.

[0040] exist Figure 1 In the decoding matrix, the elements at high grayscale positions are non-zero real numbers. It can be seen that each row of the decoding matrix corresponds to a data block, that is, each row of the decoding matrix is ​​multiplied by the column matrix on the right (each element of the row is used as the coefficient of each block in the column matrix for linear operation) to obtain a data block. For example, the first row of the decoding matrix (each element is x1, x2, x3, x4, x5) corresponds to D1, and:

[0041] D1= x1*D2+x2*D3+x3*D5+x4*C1+x5*C3 Formula 1

[0042] See also Figure 2 In the prior art, when an abnormality occurs in the data block D1 stored in the target node (which can be any node in the distributed cluster), the target node needs to pull these blocks from the node storing the normal blocks of the corresponding data object to the local node, execute the calculation process of formula 1, and thus recover the abnormal data block D1. In an embodiment of the present invention, the node selected by the target node for performing EC decoding to recover the abnormal data block and storing the normal blocks of the corresponding data object is called a decoding node. In the above method, the transmission of each block requires a large amount of network resources of the target node, including the decoding process of the multiplication and addition operations shown in formula 1, which requires a large amount of computing resources of the target node. In particular, when multiple data blocks of the target node are abnormal at the same time, it will cause huge pressure on the performance of the target node, thereby affecting the normal read and write functions of the target node, and further affecting the availability of the distributed cluster.

[0043] The concept of the present invention is to distribute the aforementioned block transmission and computation steps of the target node to each decoding node, thereby avoiding the high network overhead and CPU usage of a single node. In this embodiment of the present invention, a node storing a block of a data object locally stores its own node identifier (e.g., IP address, domain name, etc.), the identifier of the data object, the identifier of the block (e.g., D1 to D5, C1 to C3, etc.), the identifiers of the other blocks of the data object and the identifiers of the nodes storing these blocks, the corresponding encoding matrix (including the correspondence between each row and block), and the relevant computational logic for EC decoding. This information facilitates the execution of the EC decoding steps.

[0044] It should be pointed out that, in the absence of conflict, the embodiments of the present invention and the technical features therein may be combined with each other.

[0045] Figure 3 FIG. 1 is a schematic diagram of the principle of the EC decoding system of the distributed cluster in an embodiment of the present invention. Figure 3 As shown, the EC decoding system of the distributed cluster according to the embodiment of the present invention includes nodes of the distributed cluster. The distributed cluster uses the EC encoding method to determine multiple blocks of each data object for storage. Figure 4 This is a schematic diagram of the main steps of the EC decoding method of the distributed cluster in an embodiment of the present invention, which is executed by the target node. Figure 3 and Figure 4 The EC decoding method of the distributed cluster according to the embodiment of the present invention may perform the following steps:

[0046] Step S401: In response to an exception occurring in a data block of any data object stored in a target node, the target node obtains identifiers of multiple decoding nodes storing blocks of the data object.

[0047] In practical applications, when a data block of a data object stored by a target node in a distributed cluster experiences an anomaly, the target node first obtains the identifiers of multiple decoding nodes storing the blocks of the data object. It is understood that the blocks of the data object stored by each decoding node are in a normal state, and the number of such decoding nodes is equal to the total number n of data blocks formed by the data object.

[0048] In an embodiment of the present invention, the target node can determine each decoding node in the following manner. Because the target node already has the identifiers of other nodes that store the corresponding data object's blocks, it can communicate with other nodes that store the data object's blocks in a preset order to detect whether the blocks stored by the communicating nodes are normal (this process can be called liveness detection). When the number of nodes storing normal blocks reaches n, the current n detected nodes can be used as decoding nodes. The above order can be in ascending or descending order of node identifiers, or in ascending order of network distance between each node and the target node.

[0049] Step S402: The target node sends a decoding request to the decoding node according to the above identifier, so that the decoding node performs calculations based on the blocks of the data object stored locally.

[0050] In this step, the target node may first send a decoding request to one of the plurality of decoding nodes. The decoding request is transmitted in a point-to-point manner among the plurality of decoding nodes until each decoding node receives the decoding request.

[0051] In actual application, after determining each decoding node, the target node sends a decoding request to one of the multiple decoding nodes according to the identifier of the decoding node; after receiving the decoding request, the decoding node in the multiple decoding nodes sends a decoding request to a decoding node that has not received the decoding request, until each decoding node receives the decoding request. That is, the target node first sends a decoding request to a decoding node; after receiving the decoding request, the decoding node sends a decoding request to another decoding node that has not received the decoding request (the sending method can be forwarding the decoding request it received, or reassembling the decoding request for sending. The "another decoding node that has not received the decoding request" can be selected based on the preset order of the node identifier, or based on the network distance between the nodes, for example, selecting the decoding node with the smallest network distance to send the decoding request); after the other decoding node receives the decoding request, it sends a decoding request to another decoding node that has not received the decoding request (the sending method can be forwarding the decoding request it received, or reassembling the decoding request for sending). Repeat the above point-to-point communication process, and ensure that a decoding node receives at most one decoding request, until each decoding node receives the decoding request. Through the above point-to-point chain communication mechanism, each decoding node only receives one decoding request and sends at most one decoding request, achieving a balance in the number of requests among each node.

[0052] See also Figure 3, it can be seen that based on the above sending mechanism of decoding requests, each decoding node forms a chain structure connected head to tail. Among multiple decoding nodes, the node that receives the decoding request sent by the target node can be called the head node (such as decoding node 1), and the node that does not send a decoding request after receiving the decoding request can be called the tail node (such as decoding node 5). The node that sends a decoding request to any decoding node can be called the upstream node of the decoding node, and the node that receives the decoding request sent by any decoding node can be called the downstream node of the decoding node. For example, decoding node 1 is the upstream node of decoding node 2, and decoding node 2 is the downstream node of decoding node 1. There can be an intermediate node between the head node and the tail node (in this case, the number of decoding nodes is greater than 2), or there may be no intermediate node (in this case, the number of decoding nodes is equal to 2, and the head node is the upstream node of the tail node).

[0053] It should be noted that each of the above decoding requests indicates the identifier of the data object and the identifier of the block of the data object stored by each decoding node. In this way, the decoding node that receives the decoding request can locate the corresponding encoding matrix through the identifier of the data object, and determine the rows that need to be retained in the encoding matrix through the block identifier selected by the target node (that is, the identifier of the block of the data object stored by each decoding node), and then calculate the decoding matrix. The above "indication" can be carried directly in the decoding request, or it can be carried in the decoding request. The decoding node determines "the identifier of the data object and the identifier of the block of the data object stored by each decoding node" based on this information. For example, the decoding request carries the identifier of each decoding node. Since the decoding node that receives the decoding request has pre-stored the correspondence between each relevant node and the block, it can determine the corresponding block identifier based on the decoding node identifier carried by the decoding request. In this case, the decoding request also indicates "the identifier of the block of the data object stored by each decoding node."

[0054] Preferably, the decoding request can further indicate the identifier of the abnormal data block stored by the target node and the identifier of the decoding node that has not currently received the decoding request, and the meaning of the above "indication" is the same as the explanation in the previous paragraph. The "identifier of the abnormal data block stored by the target node" helps the decoding node that receives the decoding request to determine which row of the decoding matrix needs to be used for EC decoding, and the "identifier of the decoding node that has not currently received the decoding request" is used to make the decoding node that receives the decoding request clear whether it needs to send a decoding request later and the optional node to send the decoding request. If the decoding node that receives the decoding request determines from the decoding request that there are still multiple decoding nodes that have not received the decoding request, it can select a decoding node to send the decoding request according to the preset order of the node identifiers (for example, ascending or descending order), for example, select the decoding node with the smallest node identifier to send the decoding request, or select a decoding node to send the decoding request according to the ascending order of the network distance, for example, select the decoding node with the smallest network distance to send the decoding request.

[0055] After receiving a decoding request, each decoding node performs calculations based on the EC decoding matrix and the blocks of the data object stored at that decoding node. According to the previously described EC algorithm principles, the EC decoding matrix is ​​obtained by each decoding node based on the pre-stored EC encoding matrix used by the data object and the identifiers of the blocks of the data object stored by each decoding node as indicated by the received decoding request. Specifically, each decoding node first determines the corresponding encoding matrix using the data object identifier indicated in the decoding request. Then, based on the n block identifiers indicated in the decoding request, it selects n rows from the encoding matrix to form a square matrix. The inverse matrix is ​​calculated to form the decoding matrix. In this way, each decoding node performs decoding calculations based on the EC decoding matrix and the locally stored blocks. The target node then obtains the recovered abnormal data block (i.e., the normal data block recovered from the abnormal data block at the target node) based on the final calculation results of each decoding node.

[0056] The following describes the decoding calculation process of each decoding node and the abnormal data block recovery process in the embodiment of the present invention.

[0057] In one aspect, after receiving a decoding request, any decoding node determines the row vector corresponding to the abnormal data block from the EC decoding matrix, determines the target element from the row vector based on the arrangement position of the block stored by the any decoding node among the blocks stored by multiple decoding nodes, multiplies the block stored by the any decoding node by the target element, and obtains the local calculation result of the any decoding node.

[0058] The principle of the above steps is as shown in the above formula 1:

[0059] D1=x1*D2+x2*D3+x3*D5+x4*C1+x5*C3

[0060] The abnormal data block D1 can be obtained by the blocks D2, D3, D5, C1, and C3 of the decoding node through the above linear operations (including multiplication and addition operations). In the prior art, the above linear operations are all completed by the node where D1 is located. In order to achieve a balance between the amount of data transmission and the amount of calculation at each node, the embodiment of the present invention proposes that each decoding node complete the corresponding multiplication operation, and each decoding node completes the corresponding addition operation according to the position of the node in the chain structure. For the multiplication operation, that is, the decoding node where D2 is located completes x1*D2 (x1 is the target element of the D2 node), the decoding node where D3 is located completes x2*D3 (x2 is the target element of the D3 node), the decoding node where D5 is located completes x3*D5 (x3 is the target element of the D5 node), the decoding node where C1 is located completes x4*C1 (x4 is the target element of the C1 node), and the decoding node where C3 is located completes x5*C3 (x5 is the target element of the C3 node). The result of the multiplication operation is the local calculation result of each decoding node.

[0061] For any decoding node, after receiving a decoding request, it needs to determine the corresponding target element and then multiply it by the local block to obtain the local calculation result. The target element can be determined using the aforementioned method, namely, first determining the row vector corresponding to the abnormal data block from the EC decoding matrix. For example, if the abnormal data block is D1, then its corresponding row vector is the first row of the decoding matrix. Then, the target element is determined from the row vector based on the arrangement position of the block stored by the decoding node among the blocks stored by multiple decoding nodes. That is, the target element is the element of the row vector corresponding to the position of the block of the decoding node in the column vector of the decoding process. Continuing with the above example, if the decoding node is the node where D2 is located, since D2 is the first position in the column vector, its target element is the first element of the first row of the decoding matrix. If the decoding node is the node where C1 is located, since C1 is the fourth position in the column vector, its target element is the fourth element of the first row of the decoding matrix.

[0062] On the other hand, in addition to the multiplication operation, the addition operation in formula 1 needs to be completed, that is, the addition of the local calculation results of each decoding node. In practical applications, the above addition operation can be completed by some decoding nodes, or by some decoding nodes in collaboration with the target node, or by the target node independently. More preferably, the above addition operation can be distributed to each decoding result according to the position of each decoding node in the chain structure, and the target node only receives the final calculation result of all decoding nodes, that is, the recovered abnormal data block, thereby achieving a more balanced distribution of decoding workload. The specific steps are as follows:

[0063] After the tail node obtains the local calculation result, it returns the local calculation result as the final calculation result of the tail node to the upstream node of the tail node. For example, Figure 3 The tail node in the decoding node 5 returns the local calculation result (x5*C3, the decoding result of one node) to the decoding node 4 as its final calculation result.

[0064] In the case of an intermediate node, after receiving the final calculation result returned by the downstream node of the intermediate node, any intermediate node adds the final calculation result to the local calculation result of the intermediate node, and returns the result of the addition as the final calculation result of the intermediate node to the upstream node of the intermediate node. In the above steps, since the intermediate node needs to receive the final calculation result of its downstream node before performing the addition operation, the direction of the addition operation performed by each decoding node is the opposite direction of the request direction, that is, Figure 3 The order is decoding node 5, decoding node 4, decoding node 3, decoding node 2, and decoding node 1.

[0065] For example, Figure 3 After receiving the final calculation result returned by its downstream node (i.e., the local calculation result x5*C3 of decoding node 5), decoding node 4 adds the final calculation result x5*C3 to the local calculation result x4*C1 of decoding node 4, and returns the added result (x4*C1+x5*C3, the decoding results of the two nodes) as the final calculation result of decoding node 4 to decoding node 3.

[0066] After receiving the final calculation result of its downstream node (i.e., x4*C1+x5*C3 calculated by decoding node 4), decoding node 3 adds the final calculation result x4*C1+x5*C3 to the local calculation result x3*D5 of decoding node 3, and returns the added result (x3*D5+x4*C1+x5*C3, the decoding results of the three nodes) as the final calculation result of decoding node 3 to decoding node 2.

[0067] After receiving the final calculation result of its downstream node (i.e., x3*D5+x4*C1+x5*C3 calculated by decoding node 3), decoding node 2 adds the final calculation result x3*D5+x4*C1+x5*C3 to the local calculation result x2*D3 of decoding node 2, and returns the added result (x2*D3+x3*D5+x4*C1+x5*C3, the decoding results of four nodes) as the final calculation result of decoding node 2 to decoding node 1.

[0068] After receiving the final calculation result of its downstream node, the first node adds the final calculation result to the local calculation result of the first node to obtain the recovered abnormal data block. Figure 3After receiving the final calculation result of its downstream node (x2*D3+x3*D5+x4*C1+x5*C3 calculated by decoding node 2), the first node - decoding node 1 adds the final calculation result x2*D3+x3*D5+x4*C1+x5*C3 to the local calculation result x1*D2 of decoding node 1. The addition result x1*D2+x2*D3+x3*D5+x4*C1+x5*C3 is the recovered abnormal data block.

[0069] When there are only the head node and the tail node but no intermediate nodes, the head node is the upstream node of the tail node. After receiving the final calculation result of the tail node (that is, the local calculation result of the tail node), the head node adds the final calculation result to the local calculation result of the head node to obtain the recovered abnormal data block.

[0070] As can be seen, the final calculation result of the tail node is the local calculation result of the tail node, the final calculation result of any intermediate node is the sum of the local calculation result of the intermediate node and the final calculation result of the node downstream of the intermediate node, and the final calculation result of the first node is the recovered abnormal data block. The local calculation result of any decoding node is the product of the block stored by the decoding node and the target element. The target element is determined by the decoding node from the row vector based on the arrangement position of the locally stored block among the blocks stored by the multiple decoding nodes above. The above row vector is the row vector corresponding to the abnormal data block determined by any of the above decoding nodes from the EC decoding matrix after receiving the decoding request. The above EC decoding matrix is ​​obtained by any of the above decoding nodes based on the pre-stored EC encoding matrix used by the data object and the identifier of the block of the data object stored by each decoding node indicated by the decoding request.

[0071] Step S403: The target node obtains the abnormal data block restored based on the final calculation result of each decoding node.

[0072] In this step, the head node may return the recovered abnormal data block to the target node, thereby completing EC decoding.

[0073] Through the above steps, the entire process of EC decoding can be completed. Figure 3As shown, during the decoding process, after the target node sends a decoding request to decoding node 1, decoding node 1 selects decoding node 2 according to a preset rule to send a decoding request. Decoding node 2 selects decoding node 3 according to a preset rule to send a decoding request. Decoding node 3 selects decoding node 4 according to a preset rule to send a decoding request. Decoding node 4 selects decoding node 5 according to a preset rule to send a decoding request, thus forming a chain structure of decoding node 1, decoding node 2, decoding node 3, decoding node 4, and decoding node 5. After receiving the decoding request, each decoding node can perform multiplication operations in parallel to obtain local calculation results. When performing addition operations, responses are returned in the opposite direction, that is, decoding node 5, decoding node 4, decoding node 3, decoding node 2, and decoding node 1, to orderly transmit the decoding results, ultimately recovering the abnormal data block in the target node. In this way, after comprehensively considering the EC decoding computational load and data transmission volume, the workload of each decoding node is optimally distributed, ensuring that the performance of a single node is not significantly affected by EC decoding, and ensuring the high availability of the distributed cluster.

[0074] In addition, if a check block of a certain node is abnormal and needs to be restored, the node first determines whether the data blocks of the corresponding data object are normal. If so, these data blocks can be pulled locally and directly restored using the elements of the corresponding row of the encoding matrix. If it is determined that an abnormal data block exists, the node can communicate with the node where the abnormal data block is located so that the node where the abnormal data block is located executes the EC decoding method of the embodiment of the present invention to restore the abnormal data block. Thereafter, the node where the abnormal check block is located pulls the data blocks in a normal state locally and uses the elements of the corresponding row of the encoding matrix to restore the check block.

[0075] Finally, if the target node has abnormal data blocks of multiple data objects that need to be recovered, the relevant information of each data object in the decoding request can be arranged according to the disk storage order of the multiple data objects on the target node, so that the abnormal data blocks of the multiple data objects can be recovered in an orderly manner.

[0076] In the technical solution of the embodiment of the present invention, the network resources required for data transmission and the computing resources required for EC decoding are distributed between the target node and each decoding node, avoiding the high network overhead and CPU usage of a single node, while improving the EC decoding efficiency and reducing the recovery time of the data block.

[0077] Figure 5This is a schematic diagram of the main steps of the EC decoding method of a distributed cluster executed by a decoding node in an embodiment of the present invention. Since the specific details have been described above, only a brief description is given here. The distributed cluster uses an erasure code EC encoding method to determine the blocks of a data object for storage. The blocks include: data blocks and check blocks. The method is executed by multiple decoding nodes in the distributed cluster that store blocks of the same data object, such as Figure 5 The specific steps are as follows:

[0078] Step S501: When an exception occurs in the data block of the data object stored in the target node of the distributed cluster, a decoding request sent by the target node is received; Step S502: Calculation is performed according to the decoding request and the blocks of the data object stored locally by the decoding node, and a recovered abnormal data block is obtained based on the final calculation result of the decoding node; Step S503: The recovered abnormal data block is returned to the target node.

[0079] In an embodiment of the present invention, the blocks of the data object stored by each decoding node are in a normal state; and the receiving of the decoding request sent by the target node includes: based on one decoding node receiving the decoding request sent by the target node, transmitting the decoding request in a point-to-point manner within the multiple decoding nodes until each decoding node receives the decoding request.

[0080] In a specific application, the multiple decoding nodes form a chain structure, and the number of the multiple decoding nodes is equal to the total number of data blocks formed by dividing the data object; when the total number of data blocks is equal to 2, the multiple decoding nodes include a head node and a tail node; when the total number of data blocks is greater than 2, the multiple decoding nodes include a head node, an intermediate node and a tail node.

[0081] In actual applications, the calculation is performed based on the decoding request and the blocks of the data object stored locally at the decoding node, including: for the tail node, returning the local calculation result of the tail node as the final calculation result of the tail node to the upstream node of the tail node; for any intermediate node, returning the sum of the local calculation result of the intermediate node and the final calculation result of the downstream node of the intermediate node as the final calculation result of the intermediate node to the upstream node of the intermediate node; for the head node, adding the local calculation result of the head node and the final calculation result of the downstream node of the head node to obtain the recovered abnormal data block.

[0082] Preferably, the decoding request indicates: the identifier of the data object, the identifier of the block of the data object stored by each decoding node, and the identifier of the abnormal data block stored by the target node; and any decoding node determines the local calculation result through the following steps: in response to receiving the decoding request, determining an EC decoding matrix based on a pre-stored EC encoding matrix used by the data object and the identifier of the block of the data object stored by each decoding node indicated by the decoding request; determining a row vector corresponding to the abnormal data block from the EC decoding matrix, and determining a target element from the row vector based on the arrangement position of the block stored by the any decoding node in the blocks stored by the multiple decoding nodes; and multiplying the block stored by the any decoding node by the target element to obtain the local calculation result of the any decoding node.

[0083] As a preferred solution, returning the recovered abnormal data block to the target node includes: returning the recovered abnormal data block to the target node based on the first node.

[0084] In addition, in an embodiment of the present invention, each node storing the blocks of the data object pre-stores: the identifiers of other nodes storing the blocks of the data object, and the decoding request further indicates the identifiers of the decoding nodes that have not currently received the decoding request; in the case where the decoding request involves multiple data objects, the information of the multiple data objects is arranged in the decoding request according to the disk storage order of the multiple data objects on the target node.

[0085] The present invention further provides a distributed cluster EC decoding system, see Figure 3 . The distributed cluster uses an erasure code (EC) encoding method to determine the blocks of the data object for storage, and the blocks include: data blocks and check blocks. The system includes a target node and a decoding node that store the blocks of the same data object; wherein, when the target node determines that an abnormality occurs in the data block of the data object stored locally, it obtains the identifiers of multiple decoding nodes and sends a decoding request to the decoding node based on the identifiers; after receiving the decoding request, the decoding node performs calculations based on the decoding request and the blocks of the data object stored locally at the decoding node, obtains a recovered abnormal data block based on the final calculation result of the decoding node, and returns the recovered abnormal data block to the target node. Since the specific execution details have been explained above, only a brief explanation is given here.

[0086] In practical applications, when a target node determines that a data block of any locally stored data object has an anomaly, the target node obtains the identifiers of multiple decoding nodes storing the data object's blocks. Each decoding node stores a normal block of the data object, and the number of decoding nodes equals the total number of data blocks formed by the data object. The target node then sends a decoding request to one of the multiple decoding nodes based on the obtained identifier. Upon receiving the decoding request, each decoding node sends a decoding request to a decoding node that has not yet received a decoding request, until each decoding node receives a decoding request. Each decoding request indicates the identifier of the data object and the identifiers of the blocks of the data object stored by each decoding node. Upon receiving the decoding request, each decoding node performs calculations based on the EC decoding matrix and the blocks of the data object stored by that decoding node. The target node then recovers the anomaly based on the final calculation results of each decoding node. The EC decoding matrix is ​​obtained by each decoding node based on a pre-stored EC encoding matrix used by the data object and the identifiers of the blocks of the data object stored by each decoding node as indicated by the received decoding request.

[0087] In an embodiment of the present invention, each node storing the blocks of the data object pre-stores: the identifiers of other nodes storing the blocks of the data object; the target node communicates with the other nodes storing the blocks of the data object in a preset order to detect whether the blocks stored by the communicating nodes are normal, thereby determining the identifiers of the multiple decoding nodes; wherein the order includes ascending or descending order of the node identifiers, or ascending order of the network distance from the target node.

[0088] In a specific application, each decoding request further indicates the identifier of the abnormal data block stored by the target node; after receiving the decoding request, any decoding node determines the row vector corresponding to the abnormal data block from the EC decoding matrix, determines the target element from the row vector according to the arrangement position of the block stored by the any decoding node in the blocks stored by the multiple decoding nodes, multiplies the block stored by the any decoding node by the target element, and obtains the local calculation result of the any decoding node.

[0089] Preferably, among the multiple decoding nodes: the node that receives the decoding request sent by the target node is the head node; after each decoding node receives the decoding request, the node that does not send the decoding request is the tail node; after the tail node obtains the local calculation result, the local calculation result is returned to the upstream node of the tail node as the final calculation result of the tail node; wherein, the upstream node of any decoding node is the node that sends the decoding request to the decoding node; in the case of the existence of an intermediate node, after any intermediate node receives the final calculation result of the downstream node of the intermediate node, the final calculation result is added to the local calculation result of the intermediate node, and the added result is returned to the upstream node of the intermediate node as the final calculation result of the intermediate node; wherein, the intermediate node is a decoding node other than the tail node and the head node among the multiple decoding nodes; after the head node receives the final calculation result of the downstream node of the head node, the head node adds the final calculation result to the local calculation result of the head node to obtain a recovered abnormal data block and returns it to the target node.

[0090] In addition, in an embodiment of the present invention, each decoding request further indicates the identifier of a decoding node that has not currently received a decoding request; when the decoding request involves multiple data objects, the information of the multiple data objects is arranged in the decoding request according to the disk storage order of the multiple data objects on the target node.

[0091] The present invention also provides an electronic device. The electronic device in an embodiment of the present invention includes: one or more processors; and a storage device configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the distributed cluster EC decoding method provided by the present invention.

[0092] Reference below Figure 6 , which shows a schematic structural diagram of a computer system 600 of an electronic device suitable for implementing an embodiment of the present invention. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0093] like Figure 6As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the computer system 600 are also stored in the RAM 603. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0094] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed in the storage section 608 as needed.

[0095] In particular, according to embodiments disclosed herein, the processes described in the main step diagrams above can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the main step diagrams. In the above embodiments, the computer program can be downloaded and installed from a network via the communication section 609 and / or installed from a removable medium 611. When the computer program is executed by the central processing unit 601, the above-described functions defined in the system of the present invention are performed.

[0096] It should be noted that the computer-readable medium described in the present invention may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0098] The units involved in the embodiments of the present invention may be implemented in software or hardware, and may also be provided in a processor.

[0099] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently and not be incorporated into the device. The computer-readable medium carries one or more programs, and when executed by the device, causes the device to perform the following steps: in response to an exception occurring in a data block of any data object stored by a target node, obtaining identifiers of multiple decoding nodes storing the blocks of the data object; sending a decoding request to the decoding node based on the identifiers, so that the decoding node performs calculations based on the locally stored blocks of the data object; and obtaining the abnormal data block recovered based on the final calculation results of each decoding node.

[0100] In the technical solution of an embodiment of the present invention, in a distributed cluster using erasure codes (EC) for data storage, if a data block stored by a target node experiences an anomaly, the target node first tentatively determines the identities of multiple decoding nodes storing the corresponding data object's blocks through active detection. The blocks stored by these decoding nodes are in a normal state and can be used to recover the anomaly data block. The number of decoding nodes is equal to the total number of data blocks formed by the data object segmentation, thereby satisfying the execution conditions of EC decoding. Thereafter, the target node sends a decoding request to a decoding node. After receiving the decoding request, each decoding node sends a decoding request to a decoding node that has not yet received the decoding request, until each decoding node receives the decoding request. That is, after the target node sends a decoding request to a decoding node, each decoding node sends a chain of decoding requests in a point-to-point communication mode until each decoding node receives the decoding request, and each decoding node only receives the decoding request once. After receiving the decoding request, each decoding node performs calculations based on the EC decoding matrix and the locally stored blocks. The target node ultimately obtains the anomaly data block recovered based on the final calculation results of each decoding node. In this way, the network resources required for data transmission and the computing resources required for EC decoding are distributed between the target node and each decoding node, avoiding the high network overhead and CPU usage of a single node, while improving EC decoding efficiency and reducing data block recovery time.

[0101] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A distributed cluster EC decoding method, characterized in that: The distributed cluster uses an erasure code (EC) encoding method to determine blocks of a data object for storage, wherein the blocks include: data blocks and check blocks; the method is executed by a target node in the distributed cluster, and the method includes: In response to an exception occurring in a data block of any data object stored by the target node, obtaining identifiers of multiple decoding nodes storing blocks of the data object; Sending a decoding request to the decoding node according to the identifier, so that the decoding node performs calculation according to the blocks of the data object stored locally; Obtaining an abnormal data block recovered based on the final calculation result of each decoding node; The multiple decoding nodes form a chain structure, and the number of the multiple decoding nodes is equal to the total number of data blocks formed by segmenting the data object; when the total number of data blocks is equal to 2, the multiple decoding nodes include a head node and a tail node; when the total number of data blocks is greater than 2, the multiple decoding nodes include a head node, an intermediate node and a tail node; for the tail node, the local calculation result of the tail node is returned to the upstream node of the tail node as the final calculation result of the tail node; for any intermediate node, the sum of the local calculation result of the intermediate node and the final calculation result of the downstream node of the intermediate node is returned to the upstream node of the intermediate node as the final calculation result of the intermediate node; for the head node, the local calculation result of the head node is added to the final calculation result of the downstream node of the head node to obtain the recovered abnormal data block.

2. The method according to claim 1, characterized in that The blocks of the data object stored by each decoding node are in a normal state, and each node storing the blocks of the data object pre-stores: the identifiers of other nodes storing the blocks of the data object; Furthermore, obtaining the identifiers of multiple decoding nodes storing the blocks of the data object includes: Communicate with other nodes that store the blocks of the data object in a preset order to detect whether the blocks stored by the communicating nodes are normal, thereby determining the identifiers of the multiple decoding nodes; wherein the order includes ascending or descending order of the node identifiers, or ascending order of the network distance from the target node.

3. The method according to claim 1, characterized in that The sending a decoding request to the decoding node according to the identifier includes: A decoding request is sent to one of the plurality of decoding nodes; wherein the decoding request is transmitted among the plurality of decoding nodes in a point-to-point manner until each decoding node receives the decoding request.

4. The method according to claim 1, wherein The decoding request indicates: an identifier of the data object, an identifier of a block of the data object stored by each decoding node, and an identifier of an abnormal data block stored by the target node; as well as, The local calculation result of any decoding node is the product of the block stored by the decoding node and the target element, where the target element is determined by the decoding node from the row vector based on the arrangement position of the locally stored block in the blocks stored by the multiple decoding nodes. The row vector is the row vector corresponding to the abnormal data block determined by any decoding node from the EC decoding matrix after receiving the decoding request. The EC decoding matrix is ​​obtained by any decoding node based on the pre-stored EC encoding matrix used by the data object and the identifier of the block of the data object stored by each decoding node indicated by the decoding request.

5. The method according to claim 1, wherein The obtaining of the abnormal data block restored based on the final calculation result of each decoding node includes: The recovered abnormal data block sent by the first node is obtained.

6. The method according to any one of claims 1 to 5, characterized in that: The decoding request further indicates an identifier of a decoding node that has not currently received a decoding request; In the case that the decoding request involves multiple data objects, information of the multiple data objects is arranged in the decoding request according to the order in which the multiple data objects are stored on the disk of the target node.

7. A distributed cluster EC decoding method, characterized in that: The distributed cluster uses an erasure code (EC) encoding method to determine blocks of a data object for storage, wherein the blocks include: data blocks and check blocks; the method is performed by multiple decoding nodes in the distributed cluster that store blocks of the same data object, and the method includes: In the case where an exception occurs to a data block of the data object stored in a target node of the distributed cluster, receiving a decoding request sent by the target node; Performing calculations according to the decoding request and the blocks of the data object stored locally at the decoding node, and obtaining recovered abnormal data blocks based on a final calculation result of the decoding node; Returning the recovered abnormal data block to the target node; The multiple decoding nodes form a chain structure, and the number of the multiple decoding nodes is equal to the total number of data blocks formed by segmenting the data object; when the total number of data blocks is equal to 2, the multiple decoding nodes include a head node and a tail node; when the total number of data blocks is greater than 2, the multiple decoding nodes include a head node, an intermediate node and a tail node; for the tail node, the local calculation result of the tail node is returned to the upstream node of the tail node as the final calculation result of the tail node; for any intermediate node, the sum of the local calculation result of the intermediate node and the final calculation result of the downstream node of the intermediate node is returned to the upstream node of the intermediate node as the final calculation result of the intermediate node; for the head node, the local calculation result of the head node is added to the final calculation result of the downstream node of the head node to obtain the recovered abnormal data block.

8. The method according to claim 7, characterized in that The blocks of the data object stored in each decoding node are in a normal state; And, the receiving a decoding request sent by the target node includes: Based on a decoding node receiving the decoding request sent by the target node, the decoding request is transmitted within the multiple decoding nodes in a point-to-point manner until each decoding node receives the decoding request.

9. The method according to claim 7, characterized in that The decoding request indicates: the identifier of the data object, the identifiers of the blocks of the data object stored by each decoding node, and the identifier of the abnormal data block stored by the target node; and any decoding node determines the local calculation result by the following steps: In response to receiving the decoding request, determining an EC decoding matrix according to a pre-stored EC encoding matrix used by the data object and an identifier of a block of the data object stored by each decoding node indicated by the decoding request; Determine a row vector corresponding to the abnormal data block from the EC decoding matrix, and determine a target element from the row vector according to an arrangement position of the block stored by any decoding node among the blocks stored by the multiple decoding nodes; The block stored in any one of the decoding nodes is multiplied by the target element to obtain a local calculation result of any one of the decoding nodes.

10. The method according to claim 7, characterized in that The returning the recovered abnormal data block to the target node includes: The recovered abnormal data block is returned to the target node based on the first node.

11. The method according to any one of claims 7 to 10, characterized in that: Each node storing the blocks of the data object pre-stores: the identifiers of other nodes storing the blocks of the data object, and the decoding request further indicates the identifiers of decoding nodes that have not currently received the decoding request; In the case that the decoding request involves multiple data objects, information of the multiple data objects is arranged in the decoding request according to the order in which the multiple data objects are stored on the disk of the target node.

12. A distributed cluster EC decoding system, characterized in that: The distributed cluster uses an erasure code (EC) encoding method to determine blocks of a data object for storage, wherein the blocks include: data blocks and check blocks. The system includes a target node and a decoding node that store blocks of the same data object; wherein, When the target node determines that an abnormality occurs in a data block of the data object stored locally, the target node obtains identifiers of multiple decoding nodes and sends a decoding request to the decoding nodes according to the identifiers; After receiving the decoding request, the decoding node performs calculation according to the decoding request and the blocks of the data object stored locally at the decoding node, obtains a recovered abnormal data block based on a final calculation result of the decoding node, and returns the recovered abnormal data block to the target node; The multiple decoding nodes form a chain structure, and the number of the multiple decoding nodes is equal to the total number of data blocks formed by segmenting the data object; when the total number of data blocks is equal to 2, the multiple decoding nodes include a head node and a tail node; when the total number of data blocks is greater than 2, the multiple decoding nodes include a head node, an intermediate node and a tail node; for the tail node, the local calculation result of the tail node is returned to the upstream node of the tail node as the final calculation result of the tail node; for any intermediate node, the sum of the local calculation result of the intermediate node and the final calculation result of the downstream node of the intermediate node is returned to the upstream node of the intermediate node as the final calculation result of the intermediate node; for the head node, the local calculation result of the head node is added to the final calculation result of the downstream node of the head node to obtain the recovered abnormal data block.

13. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images