Asynchronous data distribution method, computing device and storage medium

Abstract

The invention provides an asynchronous data distribution method, computing equipment and a storage medium. The method comprises the steps that a first node applies an erasure code algorithm to original data to generate n data fragments; wherein the first node is any network node with original data; the first node applies an error correction code algorithm to the n data fragments to generate n verification fragments; the first node broadcasts the first specified data fragment and the first specified verification fragment to other nodes, and sends the n verification fragments to the corresponding network nodes according to the first corresponding relation, so that the second node determines original data according to the received multiple data fragments and multiple verification fragments; wherein the first corresponding relation is used for indicating the corresponding relation between the data fragment and the verification fragment and the network node; the second node is any network node without original data. According to the scheme, data can be efficiently and reliably distributed in an unreliable asynchronous network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to an asynchronous data distribution method, a computing device and a storage medium. BACKGROUND

[0002] Asynchronous Data Dissemination (ADD) protocol is a core mechanism for ensuring that all network nodes can eventually reach an agreement on the same original data. The core requirement is that if more than 1 / 3 of the network nodes take the original data as input, then all network nodes should eventually output the original data.

[0003] Therefore, how to efficiently and reliably distribute data in an unreliable asynchronous network needs to be solved. SUMMARY

[0004] The present application provides an asynchronous data distribution method, a computing device and a storage medium, which can efficiently and reliably distribute data in an unreliable asynchronous network.

[0005] In a first aspect, an embodiment of the present application provides an asynchronous data distribution method, which can be executed by an asynchronous data distribution apparatus. The asynchronous data distribution apparatus can be a terminal device or a module for a terminal device, or a server or a module for a server. The present application does not limit the execution subject of the method. The method comprises the following steps: a first node applies an erasure code algorithm to original data to generate n data shards; wherein the first node is any network node that owns the original data; n is a positive integer; the first node applies an error correction code algorithm to the n data shards to generate n check shards; the first node broadcasts a first specified data shard and a first specified check shard to other nodes, and sends the n check shards to corresponding network nodes according to a first correspondence relationship, so that a second node determines the original data according to the received multiple data shards and multiple check shards; wherein the first correspondence relationship is used to indicate the correspondence relationship between the data shards and the check shards and the network nodes; the first specified data shard is a data shard corresponding to the first node; the first specified check shard is a check shard corresponding to the first node; and the second node is any network node that does not own the original data.

[0006] According to the scheme, on the one hand, the n check shards are generated according to the n data shards instead of the original data, the size of the data shards is far smaller than that of the original data, and therefore the communication overhead can be reduced, the network bandwidth is saved, and the scalability of the asynchronous data distribution method in a large-scale network is improved; on the other hand, the first node broadcasts the first specified data shard and the first specified check shard to other nodes instead of broadcasting all data shards and check shards to other nodes, and the n check shards are respectively sent to corresponding network nodes instead of being sent to each network node, and under the premise of ensuring data distribution reliability, the communication efficiency is revolutionarily improved, and the scheme is particularly suitable for a consensus layer with large node scale, asynchronous and unreliable network environment, and provides strong technical support for constructing a high-performance and highly scalable blockchain system.

[0007] In a possible implementation method, the first node determines a hash value corresponding to each data shard in the n data shards, and generates check data according to the n hash values; and the first node applies the error correction code algorithm to the check data to generate the n check shards.

[0008] According to the scheme, the first node can effectively generate the n check shards.

[0009] In a possible implementation method, the first node respectively sends the n check shards to corresponding network nodes according to a correspondence between the n network nodes and the n check shards; or, the first node respectively sends the n check shards to corresponding network nodes according to a configuration file; or, the first node respectively sends the n check shards to corresponding network nodes according to a hash algorithm; wherein, the configuration file records the correspondence between the network nodes and the check shards.

[0010] According to the scheme, the first node can determine the correspondence between the network nodes and the data shards and the check shards, and therefore can respectively send the n check shards to corresponding network nodes.

[0011] In a second aspect, the embodiments of the present application provide an asynchronous data distribution method, which can be executed by an asynchronous data distribution apparatus. The asynchronous data distribution apparatus can be a terminal device or a module for a terminal device, or a server or a module for a server. The present application does not limit the execution subject of the method. The method comprises the following steps: a second node receives a first specified data shard and a first specified check shard broadcast by a first node, and receives a second specified check shard sent by the first node; the first specified data shard is a data shard corresponding to the first node; the first specified check shard is a check shard corresponding to the first node; the second specified check shard is a check shard corresponding to the second node; the first node is any network node that owns original data, and obtains n data shards through the original data and an erasure code algorithm, and obtains n check shards through the n data shards and an error correction code algorithm; the second node is any network node that does not own the original data; n is a positive integer; the second node applies the error correction code algorithm to the received multiple check shards to generate check data; the second node checks the received multiple data shards according to the check data, and determines the original data according to the data shards that pass the check and the erasure code algorithm.

[0012] In the above scheme, on the one hand, the n check shards are generated according to the n data shards, rather than the original data. The size of the data shard is much smaller than that of the original data, so that the communication overhead can be reduced, the network bandwidth is saved, and the scalability of the asynchronous data distribution method in a large-scale network is improved. On the other hand, in the decoding stage, the multiple check shards are decoded, rather than the original data. The calculation amount is much smaller than that of decoding the original data, which greatly reduces the burden of the processing unit of the second node, accelerates the recovery speed of the original data, and improves the overall throughput and real-time performance of the system.

[0013] In a possible implementation method, the second node checks the received multiple data shards according to the check data to determine a first correct data shard; the first correct data shard refers to a data shard that satisfies the number of data shards that can generate the original data and passes the check; and the second node applies the erasure code algorithm to the first correct data shard to determine the original data.

[0014] In the above scheme, the second node checks the data shards first, which improves the accuracy of the generation of the original data.

[0015] In a possible implementation, the second node determines the second specified check shards that meet a verification requirement by receiving a plurality of second specified check shards, the verification requirement being that the number of identical second specified check shards meets a verification threshold, and broadcasts the second specified check shards that meet the verification requirement to other nodes.

[0016] The second node broadcasts the second specified check shards to other nodes, which helps each node to receive enough check shards to enable correction of the check shards, and improves the accuracy of the original data generation.

[0017] In a possible implementation, the second node selects a first number of check shards from the plurality of check shards received, and generates first check data according to the first number of selected check shards and the error correction code algorithm, the first number being used to indicate a minimum number of check shards required to generate the check data, and the second node verifies whether the first check data is correct according to other check shards, and determines the first check data as the check data if the first check data is correct, the other check shards being any check shard in the plurality of check shards that is not selected.

[0018] The second node corrects the check shards to select correct check shards to generate the check data, and then uses the check data to verify the data shards, thereby improving the correctness of the verification of the data shards.

[0019] In a third aspect, an asynchronous data distribution apparatus is provided, including a transceiver unit and a processing unit, the processing unit being configured to apply an error correction code algorithm to original data to generate n data shards, the first node being any network node that owns the original data, n being a positive integer, and the first node applying an error correction code algorithm to the n data shards to generate n check shards, the transceiver unit being configured to broadcast first specified data shards and first specified check shards to other nodes, and send the n check shards to corresponding network nodes according to a first correspondence relationship, so that a second node determines the original data according to a plurality of received data shards and a plurality of check shards, the first correspondence relationship being used to indicate a correspondence relationship between data shards and check shards and network nodes, the first specified data shards being data shards corresponding to the first node, the first specified check shards being check shards corresponding to the first node, and the second node being any network node that does not own the original data.

[0020] In a possible implementation, the processing unit is specifically configured to determine a hash value corresponding to each of the n data shards, and generate check data according to the n hash values; and the first node applies the error correction code algorithm to the check data to generate the n check shards.

[0021] In a possible implementation, the transceiving unit is specifically configured to send the n check shards to corresponding network nodes according to a correspondence between the numbers of the n network nodes and the numbers of the n check shards; or, the first node sends the n check shards to corresponding network nodes according to a configuration file, wherein the configuration file records the correspondence between the network nodes and the check shards; or, the first node sends the n check shards to corresponding network nodes according to a hash algorithm.

[0022] In a fourth aspect, an embodiment of the present application provides an asynchronous data distribution apparatus, which comprises a transceiving unit and a processing unit. The transceiving unit is configured to receive a first specified data shard and a first specified check shard broadcast by a first node, and receive a second specified check shard sent by the first node; wherein the first specified data shard is a data shard corresponding to the first node; the first specified check shard is a check shard corresponding to the first node; the second specified check shard is a check shard corresponding to a second node; the first node is any network node that owns original data, and obtains n data shards through the original data and an erasure code algorithm, and obtains n check shards through the n data shards and an error correction code algorithm; the second node is any network node that does not own the original data; n is a positive integer; and the processing unit is configured to apply the error correction code algorithm to the received multiple check shards to generate check data, check the received multiple data shards according to the check data, and determine the original data according to the data shards that pass the check and the erasure code algorithm.

[0023] In a possible implementation, the processing unit is specifically configured to check the received multiple data shards according to the check data, and determine a first correct data shard; the first correct data shard refers to a data shard that satisfies a number of conditions and passes the check, wherein the conditions are that the data shard can generate the original data; and the second node applies the erasure code algorithm to the first correct data shard to determine the original data.

[0024] In a possible implementation, the processing unit is further configured to determine a second specified check shard that satisfies a verification requirement through the received multiple second specified check shards; the verification requirement is that the number of the same second specified check shards satisfies a verification threshold; and the second specified check shard that satisfies the verification requirement is broadcast to other nodes.

[0025] In a possible implementation, the processing unit is specifically configured to select a first number of check shards from the received multiple check shards, and generate first check data according to the selected first number of check shards and the error correction code algorithm; the first number is used to indicate a minimum number of check shards required to generate the check data; the first check data is verified according to other check shards to determine whether the first check data is correct, and if so, the first check data is determined as the check data; the other check shards refer to any check shard in the multiple check shards that is not selected.

[0026] In a fifth aspect, an embodiment of the present application further provides a computing device, comprising: a memory configured to store program instructions; a processor configured to invoke the program instructions stored in the memory, and execute any method of the first aspect or the second aspect according to the obtained program instructions.

[0027] In a sixth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores computer readable instructions, and when the computer reads and executes the computer readable instructions, any method of the first aspect or the second aspect is implemented.

[0028] In a seventh aspect, an embodiment of the present application provides a computer program product, which comprises computer programs executable by a computer device, and when the programs run on the computer device, the computer device executes any method of the first aspect or the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 a flowchart of an asynchronous data distribution method provided by an embodiment of the present application; Figure 2 a flowchart of an asynchronous data distribution method provided by an embodiment of the present application; Figure 3 a flowchart of an asynchronous data distribution method provided by an embodiment of the present application; Figure 4 a structural diagram of an asynchronous data distribution apparatus provided by an embodiment of the present application; Figure 5 a structural diagram of an asynchronous data distribution apparatus provided by an embodiment of the present application. DETAILED DESCRIPTION

[0030] The application provides an asynchronous data distribution method, a computing device and a storage medium, which can efficiently and reliably distribute data in an unreliable asynchronous network. The method and device are based on the same technical concept. Since the principles of the method and device for solving problems are similar, the implementation of the device and the method can be mutually referred to, and the repeated parts will not be described again.

[0031] Some terms in the application are explained below to facilitate understanding by those skilled in the art.

[0032] (1) Distributed Ledger Technology (DLT): Distributed Ledger Technology refers to a technical system for maintaining a shared data ledger in a distributed network, mainly including blockchain, directed acyclic graph (DAG) and other decentralized data structures. All participating nodes in the DLT system jointly maintain the consistency of the same ledger, without a central authority, and ensure the authenticity and non-tamperability of data through a consensus protocol.

[0033] (2) Asynchronous Data Dissemination (ADD): The asynchronous data dissemination protocol is the core mechanism of the consensus layer of the DLT system, which is used to reliably distribute the same original data to all honest nodes in the network in an asynchronous and unreliable network environment. The core requirement is that if more than 1 / 3 of the nodes use the original data M as input, then all honest nodes should eventually output the same M. The ADD protocol usually needs to consider communication efficiency and computational efficiency.

[0034] (3) Erasure Code: Erasure code is an encoding technology that encodes the original data M into n shards, so that any k shards (k < n) can recover the original data M. The parameters (k, n) represent the encoding parameters, where k is called the information block number and n is called the total block number. The advantage of erasure code is to realize data fault tolerance through redundant shards, and to reduce the communication complexity of data distribution from O(n²) to O(n), which is the most widely used encoding scheme in existing ADD protocols.

[0035] (4) Error Correcting Code: Error correcting code is different from erasure code. It enables data itself to have error detection and correction capability, which can detect and correct errors introduced in the transmission process at the receiving end. The theoretical advantage of error correcting code is to avoid additional verification data overhead, but its decoding process has extremely high computational complexity, which is usually proportional to the size of the original data, which limits its application in large-scale data distribution. The application innovatively applies error correcting code to the encoding of lightweight check data, rather than the original data, thereby avoiding the defect of its computational overhead.

[0036] (5) Merkle Tree: Merkle Tree is a tree data structure used to verify the integrity of large data sets. Its construction method is to calculate the hash value of the data level by level and build a tree structure, the leaf node is the hash of the data block, and the upper node is the combination of the hash of the lower node. To verify the authenticity of a certain data block, only the hash value of the node and the authentication path from the node to the root of the tree (size O(log n)) are required. However, in large-scale distributed systems, attaching a Merkle authentication path to each data shard will result in a total communication overhead of O(λn² log n), which becomes a serious bottleneck for system performance.

[0037] It should be understood that "at least one" in the embodiments of the present application means one or more, and "multiple" means two or more. The "and / or" describes the association relationship between the associated objects, which means that there can be three relationships. For example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can mean a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0038] In the embodiments of the present application, the descriptions such as "when", "in the case of", "if", and "if" mean that the electronic device will make corresponding processing under certain objective circumstances, not limited by time, and does not require the electronic device to have a judgment action when implemented, nor does it mean that there are other limitations.

[0039] It should be noted that in the present application, "for indicating" can include direct indication and indirect indication. When describing that certain information is used to indicate A, it can include that the information directly indicates A or indirectly indicates A, and does not mean that A must be carried in the information. Taking the first information used to indicate the first content as an example, the first information can include the first content, or part of the first content or an identifier, index, indicator of the first content, or can include an algorithm for determining the first content, a calculation parameter, etc. The present application does not limit the way of "indicating".

[0040] In the present application, "contains / including A" can be equivalent to "contains / including A information". A information is used to indicate A.

[0041] In addition, it needs to be understood that in the description of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description and cannot be understood as indicating or implying relative importance, nor can it be understood as indicating or implying order.

[0042] In distributed ledger technology, the asynchronous data distribution protocol is the core mechanism to ensure that all honest nodes can eventually agree on the same original data (such as a new block). The core requirement is that if more than 1 / 3 of the nodes use the original message M as input, then all honest nodes should eventually output M. In order to efficiently and reliably distribute data in an unreliable asynchronous network, existing technologies generally use encoding techniques to reduce communication complexity.

[0043] Existing solutions mainly fall into two categories: one is based on erasure coding design, such as AVID protocol. This type of solution encodes the original data M into n shards, so that the receiver only needs to collect any k (k < n) shards to recover the original data, thereby reducing the communication complexity from O(n²) to O(n). However, in order to verify the authenticity of each shard and prevent malicious nodes from sending incorrect shards, additional verification data is usually introduced. The most common method is based on Merkle Tree, which attaches an authentication path of size λlog n to each shard (where λ is the length of the hash value). This results in a huge additional communication overhead, totaling O(λ n² log n), which becomes a serious bottleneck for system performance when the node size n is large.

[0044] The other is based on error correction code design. This type of solution encodes the data itself to have error correction capability, which can theoretically avoid providing separate verification data for shards. However, the decoding process of error correction code has extremely high computational complexity, usually proportional to the size of the original data |M| (e.g. O(n|M|)). When M is a large file (such as MB level), the computational resources and time overhead required for decoding are huge, which seriously affects the real-time performance of data synchronization and the processing efficiency of nodes.

[0045] In summary, there is a fundamental contradiction in existing technologies: erasure coding-based solutions have high communication efficiency but high verification overhead, while error correction code-based solutions avoid verification overhead but have extremely low computational efficiency. Both of these solutions are difficult to meet the dual requirements of low communication overhead and low computational overhead in large-scale, high-dynamic asynchronous network environments, limiting the overall performance improvement of DLT systems.

[0046] Based on this, the embodiment of the application provides an asynchronous data distribution method applied to a first node in a distributed network system. The method can be executed by an asynchronous data distribution device. The asynchronous data distribution device can be a terminal device or a module for the terminal device, or a server or a module for the server. The application does not limit the execution subject of the method. As shown in the following Figure 1 The method comprises the following steps: Step 101, the first node applies an erasure code algorithm to the original data to generate n data shards.

[0047] The first node is any network node that owns the original data. The number of data shards is the same as the number of network nodes in the distributed network system.

[0048] Optionally, the distributed network system comprises n network nodes, at least n / 3 nodes own the original data, and n is a positive integer greater than or equal to 3. It should be noted that if n cannot be divided by 3, it is rounded up, for example, when n=4, at least 2 nodes own the original data.

[0049] Step 102, the first node applies an error correction code algorithm to the n data shards to generate n check shards.

[0050] The number of check shards is the same as the number of network nodes in the distributed network system.

[0051] Optionally, the first node can determine the hash value corresponding to each data shard according to the n data shards, and generate check data according to the n hash values; the first node applies an error correction code algorithm to the check data to generate n check shards.

[0052] Optionally, the first node can splice the n hash values together to generate check data, of course, the check data can also be generated according to other methods, which are not limited by the application.

[0053] Step 103, the first node broadcasts the first specified data shard and the first specified check shard to other nodes, and sends the n check shards to the corresponding network nodes according to the first correspondence, so that the second node determines the original data according to the received multiple data shards and multiple check shards.

[0054] The first correspondence is used to indicate the correspondence between the data shards and the check shards and the network nodes; the first specified data shard is the data shard corresponding to the first node; the first specified check shard is the check shard corresponding to the first node; and the second node is any network node that does not own the original data.

[0055] A possible implementation method, the first node sends n check shards to corresponding network nodes according to the correspondence between the numbers of the n network nodes and the numbers of the n check shards. Exemplarily, there are four network nodes in the distributed network system, corresponding numbers are 1, 2, 3, and 4; that is, there are node 1, node 2, node 3, and node 4, assuming that node 1 and node 2 have original data, and node 3 and node 4 do not have original data, that is, node 1 and node 2 are respectively the first node, and node 3 and node 4 are respectively the second node.

[0056] Node 1 generates data shard 1, data shard 2, data shard 3, and data shard 4; check shard 1, check shard 2, check shard 3, and check shard 4; similarly, node 2 generates data shard 1, data shard 2, data shard 3, and data shard 4; check shard 1, check shard 2, check shard 3, and check shard 4; then, according to the correspondence of the numbers, the first designated data shard of node 1 is data shard 1, and the first designated check shard is check shard 1; the first designated data shard of node 2 is data shard 2, and the first designated check shard is check shard 2; the corresponding check shard of node 3 is check shard 3; and the corresponding check shard of node 4 is check shard 4.

[0057] Then, node 1 broadcasts data shard 1 and check shard 1 to node 2, node 3, and node 4 respectively; sends check shard 2 to node 2, check shard 3 to node 3, and check shard 4 to node 4; similarly, node 2 broadcasts data shard 2 and check shard 2 to node 2, node 3, and node 4 respectively; sends check shard 1 to node 1, check shard 3 to node 3, and check shard 4 to node 4.

[0058] In another possible implementation method, the first node sends n check shards to corresponding network nodes according to a configuration file; wherein the configuration file records the correspondence between the network nodes and the data shards and the check shards. Exemplarily, the configuration file records the following content: the data shard corresponding to node 1 is data shard 2, and the check shard is check shard 2; the data shard corresponding to node 2 is data shard 3, and the check shard is check shard 3; the data shard corresponding to node 3 is data shard 4, and the check shard is check shard 4; and the data shard corresponding to node 4 is data shard 1, and the check shard is check shard 1; of course, the correspondence between the network nodes and the data shards and the check shards in the configuration file can also be other forms, which is not limited in the present application.

[0059] In another possible implementation method, the first node sends the n check shards to the corresponding network nodes according to a hash algorithm. Exemplarily, the network node identifier and the data shard identifier and the check shard identifier are mapped to the same hash ring, and the correspondence between the network nodes and the data shards and the check shards is determined according to the result of the hash ring. The identifier of each network node and the data shard and the check shard can also be hashed, and then the correspondence between the network nodes and the data shards and the check shards is obtained according to hash(x) mod n. The specific hash algorithm is not limited in the present application.

[0060] The correspondence between the network nodes and the data shards and the check shards can also be determined according to the attributes and geographical positions of the network nodes. The specific method of determining the correspondence between the network nodes and the data shards and the check shards is not limited in the present application.

[0061] Optionally, the correspondence between the network nodes and the data shards and the check shards can be determined by the distributed network system or any first node. The distributed network system or any first node can determine the correspondence between the network nodes and the data shards and the check shards before step 101, or can determine the correspondence between the network nodes and the data shards and the check shards in step 101 or step 102, which is not limited in the present application.

[0062] It is worth noting that the check shards generated by any first node are the same, and the data shards are the same.

[0063] In the above scheme, the n check shards are generated according to the n data shards, rather than the original data, and the size of the data shard is much smaller than the size of the original data, so that the communication overhead can be reduced, the network bandwidth is saved, and the scalability of the asynchronous data distribution method in a large-scale network is improved. Specifically, the distribution overhead of the check data is reduced from O(λ n² log n) in the traditional Merkle Tree scheme to O(n² λ). The object of the error correction code distribution is the check data H with a size of nλ, rather than the original data M. Therefore, the communication overhead is reduced by log n times. Taking a typical scenario n=100 as an example, log(100) ≈ 6.6, and the communication overhead can be reduced by about 6-7 times. (100) ≈ 6.6, and the communication overhead can be reduced by about 6-7 times.

[0064] In another aspect, the first node broadcasts the first specified data shard and the first specified check shard to other nodes instead of broadcasting all data shards and check shards to other nodes, and sends the n check shards to corresponding network nodes respectively instead of sending the n check shards to each network node, thereby realizing revolutionary improvement of communication efficiency under the premise of ensuring data distribution reliability, and being particularly suitable for a consensus layer with large node scale, asynchronous and unreliable network environment, and providing strong technical support for constructing a high-performance and highly scalable blockchain system.

[0065] Based on the steps 101 to 103, the asynchronous data distribution method provided by the embodiment of the application is applied to a second node in a distributed network system, and the method can be executed by an asynchronous data distribution device. The asynchronous data distribution device can be a terminal device or a module for a terminal device, or a server or a module for a server. The execution subject of the method is not limited in the application. As shown in the method, the method comprises the following steps: Figure 2 Step 201, the second node receives the first specified data shard and the first specified check shard broadcasted by the first node, and receives the second specified check shard sent by the first node.

[0066] The first specified data shard is a data shard corresponding to the first node; the first specified check shard is a check shard corresponding to the first node; the second specified check shard is a check shard corresponding to the second node; the first node is any network node that owns original data, and obtains n data shards through the original data and an erasure code algorithm, and obtains n check shards through the n data shards and an error correction code algorithm; the second node is any network node that does not own the original data; and n is a positive integer.

[0067] Step 202, the second node applies an error correction code algorithm to the received multiple check shards to generate check data.

[0068] ​Optionally, the second node selects a first number of the plurality of check shards from the received plurality of check shards, and generates first check data according to the selected first number of check shards and the error correction code algorithm; wherein the first number is used to indicate a minimum number of check shards required to generate the check data; the second node verifies whether the first check data is correct according to other check shards, and if correct, determines the first check data as the check data; wherein the other check shards refer to any check shard in the plurality of check shards that is not selected; if incorrect, determines that the second node is unable to generate the check data, and proceeds to unable to generate the original data. Illustratively, assuming that there are 4 network nodes in the distributed network system, the first number is 2, and assuming that the second node, i.e., node 3, receives 3 check shards, check shard 1, check shard 2, and check shard 3, then node 3 can generate check data according to check shard 1 and check shard 2, and then verify whether the check data is correct using check shard 3.

[0069] Optionally, if the distributed network system includes malicious nodes, the number of malicious nodes is not more than one third of the number of network nodes, and the number of the plurality of check shards received by the second node needs to be greater than or equal to 2 (the number of malicious nodes) + 1.

[0070] Optionally, when the second node determines that the first number of check shards has not been received, the second node publishes first abnormal indication information to the distributed network system or a distributed network system auxiliary message mechanism, the first abnormal indication information can be used to indicate that the second node has not received the first number of check shards, or can be used to indicate that the first node retransmits the check shards, and the specific content of the first abnormal indication information is not limited in the present application.

[0071] Step 203, the second node verifies the received plurality of data shards according to the check data, and determines the original data according to the data shards that pass the verification and the erasure code algorithm.

[0072] Optionally, the second node verifies the received multiple data fragments based on the verification data to determine the first correct data fragment. The first correct data fragment refers to a data fragment that meets the requirement of generating the required number of original data segments and passes verification. The second node applies an erasure coding algorithm to the first correct data fragment to determine the original data. For example, assuming there are four network nodes in the distributed network system, and the second node (node ​​3) receives two data fragments, data fragment 1 and data fragment 2, the first node verifies data fragment 1 and data fragment 2 based on the verification data. If both data fragment 1 and data fragment 2 are correct, then data fragment 1 and data fragment 2 are the first correct data fragments. The second node applies an erasure coding algorithm to data fragment 1 and data fragment 2 to determine the original data. If the verification data reveals that data fragment 1 or data fragment 2 is incorrect, then it is determined that the data fragment that meets the requirement of generating the required number of original data segments and passes verification is not valid; that is, node 3 cannot generate the original data.

[0073] Optionally, if the first correct data fragment does not exist, the second node publishes a second anomaly indication information to the distributed network system or the distributed network system auxiliary message mechanism. The second anomaly indication information can be used to indicate the error information of the data fragment, or it can be used to instruct the first node to resend the data fragment. This application does not limit the specific content of the second anomaly indication information.

[0074] The above scheme, on the one hand, generates n verification fragments based on n data fragments rather than the original data. The size of the data fragments is much smaller than the size of the original data, thus reducing communication overhead, saving network bandwidth, and improving the scalability of asynchronous data distribution methods in large-scale networks.

[0075] On the other hand, in the decoding stage, multiple parity segments are decoded instead of the original data. This significantly reduces the computational load compared to decoding the original data, greatly alleviating the burden on the processing unit of the second node, accelerating the recovery of the original data, and improving the overall throughput and real-time performance of the system. Specifically, this application reduces the computational complexity of the decoding stage from O(n|M|) to O(n²λ) when directly applying error-correcting codes to the original data. Since the object of error-correcting code decoding is H of size nλ, its computational load is much less than decoding original data of size |M|. This is useful for processing 1MB (|M| ≈ 1...) Taking a file of bytes (n=100, λ=32 bytes) as an example, the computational complexity of the traditional method is approximately 1. The computational complexity of this application is approximately 3.2 × 1 The computational complexity was reduced by approximately 300 times.

[0076] In one possible implementation, after receiving the second specified verification fragment sent by the first node, step 201 further includes: the second node determining the second specified verification fragment that meets the verification requirements through the received multiple second specified verification fragments; the number of second specified verification fragments with the same verification requirements meets the verification threshold; and the second node broadcasts the second specified verification fragment that meets the verification requirements to other nodes.

[0077] Optionally, the verification threshold is the minimum number of verification fragments required to generate verification data.

[0078] For example, suppose there are 4 network nodes in the distributed network system and the verification threshold is 2. Node 3 receives the verification fragment 3 sent by Node 1 and Node 2 respectively. If the two verification fragments are found to be different, it is determined that the verification fragment does not meet the verification threshold. The second node publishes a third abnormal indication information to the distributed network system or the distributed network system auxiliary message mechanism. The third abnormal indication information can be used to indicate that the verification fragment is wrong. This application does not limit the specific content of the third abnormal indication information.

[0079] Node 3 receives verification fragment 3 sent by Node 1 and Node 2 respectively. If the two verification fragments are identical, it determines that the verification fragment meets the verification threshold, and Node 3 broadcasts verification fragment 3 to other nodes. At this time, Node 4 can receive verification fragment 3; simultaneously, when Node 4 determines that verification fragment 4 meets the verification threshold, it also broadcasts verification fragment 4 to other nodes, and Node 3 can receive verification fragment 4. At this point, Node 3 contains verification fragment 1, verification fragment 2, verification fragment 3, and verification fragment 4; Node 4 contains verification fragment 1, verification fragment 2, verification fragment 3, and verification fragment 4.

[0080] The following is combined Figure 3 The specific implementation process of steps 101 to 103 and steps 201 to 203 will be described using a specific embodiment.

[0081] like Figure 3 As shown, assume a distributed ledger network contains 4 nodes (denoted as Node1, Node2, Node3, and Node4), where Node1 and Node2 are the first nodes, Node3 is the second node, and Node4 is a malicious node (since it contains malicious data, it can be considered a first node); the number of malicious nodes, f, does not exceed one-third of the network nodes, i.e., f ≤ 1. The distributed network system configuration parameters are: Total number of nodes: n = 4; Erasure code parameters: (f+1, n)-erasure code, i.e. (2, 4)-erasure code, requires at least k=2 data fragments to recover the original data M; where the first parameter f+1 represents the minimum number of data fragments required to recover the original data, and the second parameter n represents the number of data fragments generated.

[0082] Error correction code parameters: (f+1, f, n)-error-correcting code, i.e. (2, 1, 4)-error-correcting code; where the first parameter f+1 and the third parameter n have the same meaning as above, and the second parameter f represents the number of malicious nodes that can be tolerated.

[0083] Assume the original data is M = "BlockData_123" (assuming it is represented by a byte sequence), the finite field is GF(256) (byte field), and the primitive α = 2 is used.

[0084] Phase 1: Step 101 above, the first node applies erasure coding algorithm to the original data to generate n data fragments, including: when Node1 and Node2 receive the original data M = "BlockData_123", they respectively apply (2, 4) erasure coding (such as Reed-Solomon code) to encode M.

[0085] Optionally, the malicious node can apply erasure coding algorithms to the malicious data to generate n malicious data fragments, or it can directly generate n malicious data fragments; this application does not limit the method by which the malicious node generates malicious data fragments.

[0086] For example, the specific processing procedure is as follows: The first node divides M into two data blocks M = a0||a1, and uses polynomial interpolation to obtain the polynomial corresponding to M as P(x) = a0+a1. x, choosing primitive α = 2 on GF(256), compute 4 erasure code fragments: Data shard 1: = P( ) = P(1), corresponding to data segment number 1; Data shard 2: = P(α¹) = P(2), corresponding to data segment number 2; Data shard 3: = P(α²) = P(4), corresponding to data segment number 3; Data sharding 4: = P(α³) = P(8), corresponding to data segment number 4; It is worth noting that P(x) = a0 + a1 The parameters a0 and a1 in x can be the segmented data blocks a0 and a1, or data related to the segmented data blocks a0 and a1. This application does not limit them here.

[0087] Each data shard It contains partial information about the original data M. For example, suppose the four data fragments obtained after encoding are as follows: = [0x42, 0x6C, 0x6F, ...] (16 bytes); = [0x63, 0x6B, 0x44, ...] (16 bytes); = [0x61, 0x74, 0x61, ...] (16 bytes); = [0x31, 0x32, 0x33, ...] (16 bytes); It is worth noting that any two (i.e., k=f+1) fragments can be used to recover the original data M through Lagrange interpolation. This is the core advantage of erasure coding, which reduces the communication complexity from O(n²) to O(n).

[0088] Optionally, the malicious node Node4 may also generate malicious data shards m'1, m'2, m'3, and m'4.

[0089] In the second stage, the first node in step 102 above applies an error-correcting code algorithm to the n data fragments to generate n check fragments, including: Step 2.1: Node1 and Node2 calculate the hash value of all data fragments and then concatenate them.

[0090] This application does not limit the hash algorithm. Taking the SHA-256 hash function as an example, the hash value corresponding to each data fragment is as follows: has = SHA256( = 0xA1B2C3D4... (32 bytes); has = SHA256( = 0xE5F6G7H8... (32 bytes); has = SHA256( = 0xI9J0K1L2... (32 bytes); has = SHA256( = 0xM3N4O5P6... (32 bytes); Concatenate all hash values ​​into verification data: H = has || has || has || has = 0xA1B2C3D4...E5F6G7H8...I9J0K1L2...M3N4O5P6... (Total 128 bytes).

[0091] It is worth noting that the size of the verification data H is only 4 × 32 = 128 bytes, which is much smaller than the original data M (assuming M is several KB). In this application, the distributed network system avoids the huge verification overhead of M itself in traditional schemes (such as O(λ n² log n) of Merkle Tree) by "outsourcing" the verification responsibility to the lightweight verification data.

[0092] Step 2.2: After obtaining the verification data, Node1 and Node2 treat the verification data H = h1||h2 itself as the "data" to be encoded, and apply (2, 1, 4)-error correction code to H for encoding.

[0093] For example, let the polynomial corresponding to H be Q(x) = h0 + h1 x, also using primitive α = 2, calculate 4 check slices: Verify fragment 1: = Q( ) = Q(1); Verify fragment 2: = Q(α¹) = Q(2); Verify fragment 3: = Q(α²) = Q(4); Verify fragment 4: = Q(α³) = Q(8); For example: = [0xF1, 0xE2, 0xD3, ...] (128 bytes, i.e., a transformation result of H); = [0xC4, 0xB5, 0xA6, ...] (128 bytes); = [0x97, 0x88, 0x79, ...] (128 bytes); = [0x6A, 0x5B, 0x4C, ...] (128 bytes); It is worth noting that Q(x) = h0 + h1 The parameters h0 and h1 in x can be the segmented data blocks h1 and h2, or data related to the segmented data blocks h1 and h2. This application does not limit them here.

[0094] The purpose of error-correcting codes: These parity segments have error-correcting capabilities. Even if multiple parity segments are lost, the complete H can be recovered from the received parity segments through the decoding protocol.

[0095] Optionally, the malicious node Node4 may also generate malicious verification shards e'1, e'2, e'3, and e'4. This application does not limit the method by which the malicious node generates malicious verification shards.

[0096] In the third stage, step 103 above, the first node broadcasts the first designated data fragment and the first designated checksum allocation to other nodes, and sends the other checksum fragments to the designated network nodes respectively, including: Step 3.1: The first node sets the initial flag Retr = 0, indicating that the node has not yet completed a round of data distribution; the first node, namely Node1 and Node2, broadcasts its specified data fragments to other nodes, and the malicious node Node4 also broadcasts its specified data fragments to other nodes.

[0097] Specifically, Node1 broadcasts data fragment 1 (m1) to Node2, Node3, and Node4; Node2 broadcasts data fragment 2 (m2) to Node1, Node3, and Node4; and Node4 broadcasts data fragment 4 (m'4) to Node1, Node2, and Node3. Of course, the first nodes may not broadcast data fragments to each other, and this application does not impose any restrictions on this.

[0098] For example, such as Figure 3 As shown, Node1 broadcasts the message (Start, m1) to all nodes, Node2 broadcasts the message (Start, m2) to all nodes, and Node4 broadcasts the message (Start, m'4) to all nodes. The START flag indicates that this is the first distribution attempt. After receiving this message, all nodes know that a new data distribution process has started.

[0099] Communication complexity analysis: This step of broadcasting generates O(n) messages, each message being |m1| (approximately 16 bytes) in size, with a total overhead of O(n|m|).

[0100] Step 3.2: The first node and the malicious node each send n verification fragments to the designated network node.

[0101] For example, Node1 sends (Call, e2) to Node2; Node1 sends (Call, e3) to Node3; Node1 sends (Call, e4) to Node4; Node2 sends (Call, e1) to Node1; Node2 sends (Call, e3) to Node3; Node2 sends (Call, e4) to Node4; Node4 sends (Call, e'1) to Node1; Node4 sends (Call, e'2) to Node2; Node4 sends (Call, e'3) to Node3.

[0102] Optionally, multiple first nodes can send n parity fragments to designated network nodes in parallel.

[0103] This targeted distribution (rather than a full broadcast) ensures that each node obtains the correct parity fragment belonging to it, while avoiding redundant broadcasts. For example, the parity fragment obtained by each node is the parity fragment corresponding to the node's own sequence identifier; for instance, nodes 1, 2, 3, and 4 obtain the 1st, 2nd, 3rd, and 4th parity fragments, respectively.

[0104] In the fourth stage, each node broadcasts its corresponding designated checksum fragment.

[0105] Optionally, when the first node receives the verification fragments sent by the second node, and receives f+1=2 verification fragments, the second node can confirm that these verification fragments have been recognized by a sufficient number of nodes.

[0106] For example, if Node3 receives a verification fragment 3 from Node1 and Node2 respectively, and receives a malicious verification fragment 3 from Node4, then Node3 broadcasts the verification fragments it received to all nodes.

[0107] Optionally, when Node3 receives 3 check fragments, it compares the 3 check fragments and determines that the malicious check fragment 3 is different from the other check fragments, and then discards the malicious check fragment 3.

[0108] Node1 broadcasts the message (Check, e1) to all nodes; Node2 broadcasts the message (Check, e2) to all nodes; Node3 broadcasts the message (Check, e3) to all nodes; Node4 broadcasts the message (Check, e'4) to all nodes.

[0109] Optionally, the first nodes, namely Node1 and Node2, can broadcast their corresponding parity fragments to all nodes in the third stage, i.e., step 103 above. Then, in the fourth stage, it is not necessary to broadcast their corresponding parity fragments to all nodes again. For example... Figure 3 As shown, Node3 only needs to broadcast the message (Check, e3) to all nodes.

[0110] Optionally, the first node changes the Retr flag from 0 to 1, indicating that the node has completed the initial confirmation of a round of verification of the fragment.

[0111] In the fifth stage, in steps 201 to 203 above, the second node determines the original data based on the received multiple data fragments and n check fragments, as shown below: Step 5.1: Verify the fragment collection and error correction code decoding trigger.

[0112] Optionally, the second node receives 2f+1=3 (Check, When a message is received, it indicates that enough nodes have confirmed the validity of the verification shard, at which point error correction code decoding is triggered.

[0113] It is worth noting that since the first node already has the original data M, it will directly output M.

[0114] Step 5.2: Decode the verification data based on the error correction code.

[0115] For example, suppose Node3 receives three parity fragments, e1, e2 and e3. Node3 starts the error correction code decoding protocol and uses the received three parity fragments to recover the complete parity data H.

[0116] The specific decoding steps are as follows: Node3 has the following parity shards: e1 = Q(1), e2 = Q(2), e3 = Q(4); Construct the interpolation polynomial Q(x) using Lagrange interpolation or Gaussian elimination: Choose two points (x1, e1) = (1, e1) and (x2, e2) = (2, e2), and calculate the Lagrange basis functions: L1(x) =(x-x2) / (x1-x2) = (x-2) / (1-2) = 2-x; L2(x) =(x-x1) / (x2-x1) = (x-1) / (2-1) = x-1, The polynomial is recovered by a linear combination of basis functions: Q(x) = e1 L1(x) + e2 L2(x) = (2e1-e2) + (e2-e1)x; According to the coefficients of the polynomial, H = h0||h1 = (2e1-e2) || (e2-e1); Thus, H = has can be recovered. || has || has || has ; Verification process: Node3 uses the third parity fragment e3 = Q(4) and substitutes it into the recovered Q(x) for verification: Calculate Q(4) and compare it with e3; If they are equal, then H is correctly recovered; if they are not equal, it means that the verification fragment came from a malicious node or is corrupted, and it should be discarded. If H is successfully recovered, Node 3 now has all the hash values: has has has has .

[0117] The computational complexity of the decoding process: The object of the error-correcting code decoding is a 128-byte H, not the original KB or even MB data M. The polynomial operations and matrix inversion involved in the decoding process have a computational complexity of O(n²λ) = O(16 × 32) = O(512-byte level operations), which is far better than O(n|M|) = O(4 × several KB) = O(several KB level operations).

[0118] Step 5.3: Validation of raw data.

[0119] Node3 receives data fragments from other nodes on the network. Each node broadcasts its (Start, ...) data fragments. )information.

[0120] Suppose that the data fragments received by Node3 are as follows: Received from Node1: m1 (correct); Received from Node2: m2 (correct); Received from Node3: m'4 (erroneous data fragment sent by a malicious node).

[0121] Node3 calculates the hash value for each received data fragment: hash(m1) = 0xA1B2C3D4... → with the recovered hash Compare → Match ✓; hash(m2) = 0xI9J0K1L2... → with the recovered hash Compare → Match ✓; hash(m'4) = 0xM3N4O5P6... → with the recovered hash Comparison → Mismatch ✗.

[0122] Therefore, Node 3 selects m1 and m3 to be included in the erasure coding decoding algorithm to calculate the original data M.

[0123] Fault identification: m'4 sent by Node4 was identified as erroneous due to a hash value mismatch and was discarded. This demonstrates the strong protection this application provides against malicious nodes: even if a malicious node attempts to send forged data fragments, it will be exposed during the hash verification of H.

[0124] Step 5.4: Recovery of the original data.

[0125] Node 3 has now been confirmed to have two correct data fragments: m1 and m2. Since k = f + 1 = 2, and Node 3 already has two correct fragments, it can use erasure coding to reconstruct the original data M.

[0126] For example, using the polynomial points corresponding to the segments m1 and m2: P(1) = m1; P(4) = m2; Construct the polynomial P(x) using Lagrange interpolation: P(x) = + ·x + ... + a_{k-1}·x^(k-1), using the two conditions P(1) = m1 and P(4) = m3, the coefficients of P(x) are uniquely determined; the coefficient sequence of the polynomial P(x)[ , [, ...] represents the various parts of the original data M; the complete M = "BlockData_123" is reconstructed.

[0127] like Figure 3 As shown, Node 3 successfully recovered the original data M and used it as valid block data for subsequent processing (such as executing smart contracts, updating ledger state, etc.), thus completing this round of data distribution protocol.

[0128] Based on the same technological concept Figure 4 An asynchronous data distribution apparatus 400 provided in an embodiment of this application is illustrated by way of example. Figure 4As shown, the system includes a transceiver unit 401 and a processing unit 402. The processing unit 402 is used to apply an erasure coding algorithm to the original data to generate n data fragments. The first node is any network node that possesses the original data; n is a positive integer. The first node applies an error correction coding algorithm to the n data fragments to generate n check fragments. The transceiver unit 401 is used to broadcast a first designated data fragment and a first designated check fragment to other nodes, and to send the n check fragments to their respective network nodes according to a first correspondence, so that the second node can determine the original data based on the received data fragments and check fragments. The first correspondence is used to indicate the correspondence between data fragments, check fragments, and network nodes. The first designated data fragment is the data fragment corresponding to the first node. The first designated check fragment is the check fragment corresponding to the first node. The second node is any network node that does not possess the original data.

[0129] In one possible implementation, the processing unit 402 is specifically used to determine the hash value corresponding to each of the n data shards, and generate verification data based on the n hash values; the first node applies the error correction code algorithm to the verification data to generate the n verification shards.

[0130] In one possible implementation, the transceiver unit 401 is specifically configured to send the n verification fragments to the corresponding network nodes according to the correspondence between the n network node numbers and the n verification fragment numbers; or, the first node sends the n verification fragments to the corresponding network nodes according to a configuration file; wherein the configuration file records the correspondence between network nodes and verification fragments; or, the first node sends the n verification fragments to the corresponding network nodes according to a hash algorithm.

[0131] In one possible implementation, the transceiver unit 401 is configured to receive a first designated data fragment and a first designated check fragment broadcast by a first node, and to receive a second designated check fragment sent by the first node; wherein, the first designated data fragment is a data fragment corresponding to the first node; the first designated check fragment is a check fragment corresponding to the first node; the second designated check fragment is a check fragment corresponding to the second node; the first node is any network node possessing the original data, and obtains n data fragments through the original data and an erasure coding algorithm, and obtains n check fragments through the n data fragments and an error correction coding algorithm; the second node is any network node not possessing the original data; n is a positive integer; the processing unit 402 is configured to apply the error correction coding algorithm to the received multiple check fragments to generate check data, and verify the received multiple data fragments according to the check data, and determine the original data according to the verified data fragments and the erasure coding algorithm.

[0132] In one possible implementation, the processing unit 402 is specifically used to verify the received multiple data fragments according to the verification data to determine the first correct data fragment; the first correct data fragment refers to a data fragment that meets the requirement of generating the original data and passes the verification; the second node applies the erasure coding algorithm to the first correct data fragment to determine the original data.

[0133] In one possible implementation, the processing unit 402 is further configured to determine, through the received multiple second specified verification fragments, a second specified verification fragment that meets the verification requirements; the number of second specified verification fragments with the same verification requirements meets the verification threshold; and broadcast the second specified verification fragments that meet the verification requirements to other nodes.

[0134] In one possible implementation, the processing unit 402 is specifically configured to select a first number of check fragments from a plurality of received check fragments, and generate first check data based on the selected first number of check fragments and the error correction code algorithm; the first number is used to indicate the minimum number of check fragments required to generate the check data; verify whether the first check data is correct based on other check fragments, and if correct, determine the first check data as the check data; wherein, the other check fragments refer to any check fragment among the plurality of check fragments that have not been selected.

[0135] Based on the same technical concept, embodiments of this application provide an asynchronous data distribution device 500, which may be, for example, a computing device. Figure 5As shown, an asynchronous data distribution device 500 includes at least one processor 501 and a memory 502 connected to the at least one processor. In this embodiment, the specific connection medium between the processor 501 and the memory 502 is not limited. Figure 5 Taking the connection between processor 501 and memory 502 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0136] In this embodiment of the application, the memory 502 stores instructions that can be executed by at least one processor 501. By executing the instructions stored in the memory 502, at least one processor 501 can execute the above-described asynchronous data distribution method.

[0137] The processor 501 serves as the control center of the asynchronous data distribution device 500. It can connect to various parts of the computer device via various interfaces and lines, and performs resource configuration by running or executing instructions stored in the memory 502 and accessing data stored in the memory 502. Optionally, the processor 501 may include one or more determining units. The processor 501 may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0138] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0139] Memory 502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 502 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 502 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 502 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0140] This application also provides a computer-readable storage medium storing a computer-executable program, which is used to cause a computer to perform an asynchronous data distribution method listed in any of the above embodiments.

[0141] This application provides a computer program product, including a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform an asynchronous data distribution method listed in any of the above methods.

[0142] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0144] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0145] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0146] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An asynchronous data distribution method applied to a distributed network system comprising n network nodes, characterized in that, The method comprises: The first node applies an erasure code algorithm to the original data to generate n data shards; wherein the first node is any network node that owns the original data; n is a positive integer; The first node applies an error correction code algorithm to the n data shards to generate n check shards; The first node broadcasts a first specified data shard and a first specified check shard to other nodes, and sends the n check shards to corresponding network nodes according to a first correspondence, so that a second node determines the original data according to the received multiple data shards and multiple check shards; wherein the first correspondence indicates the correspondence between data shards and check shards and network nodes; the first specified data shard is the data shard corresponding to the first node; the first specified check shard is the check shard corresponding to the first node; and the second node is any network node that does not own the original data.

2. The method of claim 1, wherein, The first node applies an error correction code algorithm to the n data shards to generate n check shards, comprising: The first node determines the hash value corresponding to each data shard in the n data shards, and generates check data according to the n hash values; The first node applies the error correction code algorithm to the check data to generate the n check shards.

3. The method of claim 1, wherein, The first node sends the n check shards to corresponding network nodes according to the correspondence between the n network nodes and the n check shard numbers; or, The first node sends the n check shards to corresponding network nodes according to a configuration file; wherein the configuration file records the correspondence between network nodes and check shards; or, The first node sends the n check shards to corresponding network nodes according to a hash algorithm. The method comprises:

4. An asynchronous data distribution method applied to a distributed network system comprising n network nodes, characterized in that, The second node receives a first specified data shard and a first specified check shard broadcast by the first node, and receives a second specified check shard sent by the first node; wherein the first specified data shard is the data shard corresponding to the first node; the first specified check shard is the check shard corresponding to the first node; the second specified check shard is the check shard corresponding to the second node; the first node is any network node that owns the original data, and obtains n data shards through the original data and an erasure code algorithm, and obtains n check shards through the n data shards and an error correction code algorithm; the second node is any network node that does not own the original data; n is a positive integer; The second node applies the error correction code algorithm to the received multiple check shards to generate check data; The second node checks the received multiple data shards according to the check data, and determines the original data according to the data shards that pass the check and the erasure code algorithm. ​ 5. The method of claim 4, wherein, The second node checks the received multiple data shards according to the check data, and applies an erasure code algorithm to the data shard that passes the check to determine the original data, including: The second node checks the received multiple data shards according to the check data, and determines a first correct data shard; the first correct data shard refers to a data shard that meets the number of original data that can be generated and passes the check; The second node applies the erasure code algorithm to the first correct data shard to determine the original data.

6. The method of claim 4, wherein, The method further includes: The second node determines a second specified check shard that meets the verification requirement through the received multiple second specified check shards; the verification requirement is that the number of the same second specified check shards meets the verification threshold; The second node broadcasts the second specified check shard that meets the verification requirement to other nodes.

7. The method of claim 4, wherein, The second node applies the error correction code algorithm to the received multiple check shards to generate check data, including: The second node selects a first number of check shards from the received multiple check shards, and generates first check data according to the selected first number of check shards and the error correction code algorithm; the first number is used to indicate the minimum number of check shards required to generate check data; The second node verifies whether the first check data is correct according to other check shards, and if so, determines the first check data as the check data; wherein the other check shards refer to any check shard in the multiple check shards that is not selected.

8. A computing device, comprising: including: a memory for storing program instructions; a processor for invoking the program instructions stored in the memory, and executing the method according to any one of claims 1 to 3, or the method according to any one of claims 4 to 7, according to the obtained program instructions.

9. A computer-readable storage medium, characterized in that, including computer readable instructions, when the computer reads and executes the computer readable instructions, the method according to any one of claims 1 to 3 is implemented, or the method according to any one of claims 4 to 7 is implemented.

10. A computer program product, characterised in that, including a computer program executable by a computer device, when the program runs on the computer device, the computer device executes the steps of the method according to any one of claims 1 to 3, or the steps of the method according to any one of claims 4 to 7.

Images