Data processing method and device based on block chain, equipment and storage medium

Abstract

The invention provides a block chain-based data processing method and device, equipment and a storage medium. The method comprises the steps of obtaining transaction data; processing the transaction data according to a preset number of consensus nodes to obtain data blocks of the transaction data; wherein the data block comprises sub-transaction blocks and verification blocks, the sub-transaction blocks represent part of transaction data, the verification blocks are used for replacing the sub-transaction blocks, and the sum of the number of the sub-transaction blocks and the number of the verification blocks is the preset number of consensus nodes; and storing the sub-transaction block and the verification block in a preset database on the block chain. By obtaining the sub-transaction block and the verification block, the transaction data can be directly stored on the block chain.

Description

Technical Field

[0001] The present application relates to blockchain technology, and in particular to a blockchain-based data processing method, device, equipment and storage medium. Background Art

[0002] The existing method of uploading large amounts of data to the chain is to store the original data in advance and only store the data summary on the chain, while maintaining a set of data storage services and their supporting disaster recovery separately, which leads to overall bloat, difficulty in maintenance, and high maintenance costs.

[0003] Therefore, how to save large amounts of data directly to the blockchain and reduce maintenance costs is an urgent problem that needs to be solved. Summary of the invention

[0004] The present application provides a blockchain-based data processing method, device, equipment and storage medium to solve the problem that large amounts of data cannot be directly saved to the blockchain.

[0005] In a first aspect, the present application provides a data processing method based on blockchain, comprising:

[0006] Get transaction data;

[0007] According to the preset number of consensus nodes, the transaction data is processed to obtain a data block of the transaction data; wherein the data block includes a sub-transaction block and a check block, the sub-transaction block represents part of the transaction data, the check block is used to replace the sub-transaction block, and the sum of the number of sub-transaction blocks and check blocks is the preset number of consensus nodes;

[0008] The sub-transaction blocks and verification blocks are stored in the preset database on the blockchain.

[0009] Among them, in this application, according to the preset number of consensus nodes, the transaction data is processed to obtain a data block of the transaction data, including:

[0010] According to the preset number of consensus nodes and based on the preset consensus algorithm, the number of sub-transaction blocks and the number of verification blocks are determined;

[0011] According to the number of sub-transaction blocks and the number of check blocks, the transaction data is processed to obtain a data block of the transaction data. In this application, the formula for determining the number of sub-transaction blocks and the number of check blocks is:

[0012] n = ɑf + 1;

[0013] Among them, n represents the preset number of consensus nodes, f represents the number of verification blocks, (ɑ-1)×(f+1) represents the number of sub-transaction blocks, and ɑ is a preset parameter.

[0014] In this application, the transaction data is processed according to the number of sub-transaction blocks and the number of check blocks to obtain a data block of the transaction data, including:

[0015] According to the number of sub-transaction blocks, the number of check blocks, and the transaction data, based on the preset first erasure coding algorithm, the data content in the sub-transaction block and the data content in the check block are determined to obtain a data block of the transaction data.

[0016] The sub-transaction blocks and verification blocks are stored in the preset database on the blockchain, including:

[0017] Determine a hash value of the sub-transaction block as a first hash value, and determine a hash value of the verification block as a second hash value;

[0018] Performing signature calculation on the first hash value to obtain first signature data, and performing signature calculation on the second hash value to obtain second signature data; wherein the first signature data represents the data content in the sub-transaction block, and the second signature data represents the data content in the verification block;

[0019] The sub-transaction block, the verification block, the first signature data, and the second signature data are stored in a preset database on the blockchain.

[0020] In this application, determining the hash value of the sub-transaction block as the first hash value includes:

[0021] Perform hash calculation on the transaction data according to the preset hash algorithm to obtain the hash value of the transaction data;

[0022] Determine the first identification data of the sub-transaction block according to the position of the sub-transaction block in the transaction data;

[0023] According to the hash value of the transaction data and the first identification data, the hash value of the sub-transaction block is determined as the first hash value.

[0024] In this application, determining the hash value of the check block as the second hash value includes:

[0025] Allocating second identification data to the check block;

[0026] According to the hash value of the transaction data and the second identification data, the hash value of the verification block is determined as the second hash value.

[0027] Among them, in this application, the method also includes:

[0028] Receive the transaction ID of the transaction to be queried, determine the number of sub-transaction blocks corresponding to the transaction to be queried, and determine the sub-transaction block corresponding to the transaction ID as the target sub-transaction block;

[0029] If the number of target sub-transaction blocks is equal to the number of sub-transaction blocks corresponding to the transaction to be queried, the target sub-transaction blocks are restored to the transaction data of the transaction to be queried according to the preset second erasure coding algorithm.

[0030] Among them, in this application, the method also includes:

[0031] If the number of target sub-transaction blocks is less than the number of sub-transaction blocks corresponding to the transaction to be queried, then determine the difference between the number of target sub-transaction blocks and the number of sub-transaction blocks corresponding to the transaction to be queried;

[0032] Obtain the check block corresponding to the transaction ID, and determine the target check block from the check blocks corresponding to the transaction ID according to the difference;

[0033] According to the preset second erasure coding algorithm, the target sub-transaction block and the target check block are restored to the transaction data of the transaction to be queried.

[0034] In a second aspect, the present application provides a data processing device based on blockchain, comprising:

[0035] Acquisition module, used to obtain transaction data;

[0036] The obtaining module is used to process the transaction data according to the preset number of consensus nodes to obtain a data block of the transaction data; wherein the data block includes a sub-transaction block and a check block, the sub-transaction block represents part of the transaction data, the check block is used to replace the sub-transaction block, and the sum of the number of the sub-transaction blocks and the check block is the preset number of consensus nodes;

[0037] The storage module is used to store the sub-transaction blocks and the verification blocks in a preset database on the blockchain.

[0038] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0039] Memory stores computer-executable instructions;

[0040] The processor executes the computer-executable instructions stored in the memory to implement the method in the present application.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method of the present application.

[0042] The present application provides a data processing method, device, equipment and storage medium based on blockchain, which obtains transaction data; processes the transaction data according to the preset number of consensus nodes to obtain a data block of the transaction data; wherein the data block includes a sub-transaction block and a check block, the sub-transaction block represents part of the transaction data, the check block is used to replace the sub-transaction block, and the sum of the number of sub-transaction blocks and check blocks is the preset number of consensus nodes; through the preset number of consensus nodes, the sub-transaction block and the check block can be obtained according to the transaction data, so that the transaction data can be directly saved on the blockchain. The existence of the check block can improve the fault tolerance rate. By storing the sub-transaction block and the check block in the preset database on the blockchain, a large amount of data can be saved in the blockchain, improving the security of the data, achieving the goal of eliminating the need for external storage and saving storage costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] Figure 1 A flowchart of a data processing method based on blockchain provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of another blockchain-based data processing method provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a blockchain-based data processing device provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0048] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0049] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0050] In the prior art, for blockchain, only the data summary is stored on the chain, and then the actual data is stored in an independent data storage server and IPFS (InterPlanetary File System) storage cluster. Whether using an IPFS storage cluster or an independent data storage server, a set of data storage services and their supporting disaster recovery need to be maintained separately, resulting in overall bloat, difficult maintenance, and high maintenance costs. At the same time, storing the data summary and the data separately is more inclined to a centralized data management model. The data cannot be fully distributed for storage and use. Once the data service is damaged, it will become unavailable and have low risk resistance.

[0051] The present application provides a data processing method, device, equipment and storage medium based on blockchain, which obtains transaction data; processes the transaction data according to the preset number of consensus nodes to obtain a data block of the transaction data; wherein the data block includes a sub-transaction block and a check block, the sub-transaction block represents part of the transaction data, the check block is used to replace the sub-transaction block, and the sum of the number of sub-transaction blocks and check blocks is the preset number of consensus nodes; through the preset number of consensus nodes, the sub-transaction block and the check block can be obtained according to the transaction data, so that the transaction data can be directly saved on the blockchain. The existence of the check block can improve the fault tolerance rate. By storing the sub-transaction block and the check block in the preset database on the blockchain, a large amount of data can be saved in the blockchain, improving the security of the data, achieving the goal of eliminating the need for external storage and saving storage costs.

[0052] First, the terms involved in this application are explained:

[0053] Blockchain is a distributed ledger technology that ensures the security and immutability of data through decentralization and encryption algorithms. Blockchain was originally introduced as the underlying technology of Bitcoin, but its application has expanded to many other fields, such as finance, supply chain management, medical care, the Internet of Things, etc.

[0054] Error Correction Code (ECC) is an algorithm used to detect and correct errors that occur during data transmission or storage. Erasure codes are widely used in computer storage, communication, data transmission and other fields.

[0055] Figure 1 A flowchart of a data processing method based on blockchain provided in an embodiment of the present application. Figure 1 As shown, the execution subject of the method can be a server, and this embodiment does not make any special restrictions here. Figure 1 As shown, the method includes:

[0056] S101, obtaining transaction data;

[0057] Among them, transaction data refers to various information generated and recorded during the transaction process uploaded by the client, including: transaction time, transaction ID, transaction content, etc. Transaction data can be obtained in real time or at a fixed time.

[0058] S102. Process the transaction data according to the preset number of consensus nodes to obtain a data block of the transaction data; wherein the data block includes a sub-transaction block and a check block, the sub-transaction block represents part of the transaction data, the check block is used to replace the sub-transaction block, and the sum of the number of the sub-transaction blocks and the check block is the preset number of consensus nodes;

[0059] Among them, consensus nodes are a key component in distributed or blockchain networks. They participate in and help reach a consensus among all nodes in the network on a certain state or transaction. The main function of consensus nodes is to ensure the consistency and integrity of data in the network, that is, all nodes reach a consensus on a certain data state to prevent data tampering and double payments. Consensus nodes play a vital role in maintaining the security and stability of blockchain networks. Through a distributed consensus mechanism, blockchain networks can achieve trustless transactions and data management without a centralized organization.

[0060] The number of consensus nodes is pre-set. The number of consensus nodes can be the number of data blocks. A data block can include multiple sub-transaction blocks and multiple verification blocks. The number of consensus nodes can determine the total number of sub-transaction blocks and verification blocks. That is, each consensus node corresponds to a data block, which can store a sub-transaction block or a verification block.

[0061] A sub-transaction block can represent part of the transaction data. The check block is used to replace the sub-transaction block. Through the preset number of consensus nodes, the transaction data can be processed and divided into multiple sub-transaction blocks.

[0062] The check block can also be used as a redundant block, which can be used to replace the lost data to restore the original data when part of the data (i.e., the sub-transaction block) is lost or damaged. For example, if the sub-transaction block A is lost, the check block B can be used to replace the sub-transaction block A.

[0063] Among them, in the embodiment of the present application, the transaction data is processed according to the preset number of consensus nodes to obtain a data block of the transaction data, including:

[0064] According to the preset number of consensus nodes and based on the preset consensus algorithm, the number of sub-transaction blocks and the number of verification blocks are determined;

[0065] The transaction data is processed according to the number of sub-transaction blocks and the number of check blocks to obtain a data block of the transaction data.

[0066] Among them, the consensus algorithm is a mechanism used in distributed systems and blockchain networks to ensure that all nodes reach a consensus on a certain state or transaction. The main goal of these algorithms is to ensure the consistency and integrity of data, even in the presence of malicious nodes or network failures. Consensus algorithms include: Proof of Work (PoW), Proof of Stake (PoS), Delegated Proof of Stake (DPoS), Byzantine Fault Tolerance (BFT), Practical Byzantine Fault Tolerance (PBFT), Proof of Authority (PoA), Raft algorithm. In this embodiment, the preset consensus algorithm is not specifically limited.

[0067] Different consensus algorithms correspond to different theoretical fault tolerance rates, which will determine the number of sub-transaction blocks and the number of verification blocks. However, regardless of the type of consensus algorithm, the sum of the number of sub-transaction blocks and the number of verification blocks is always equal to the preset number of consensus nodes. In short, the preset consensus algorithm and the preset number of consensus nodes will affect the number of sub-transaction blocks and verification blocks.

[0068] By presetting the consensus algorithm and the number of consensus nodes, the number of sub-transaction blocks and the number of check blocks can be determined, and then the transaction data can be divided and processed according to the number of sub-transaction blocks (for example, it can be divided into sub-transaction blocks of equal size or sub-transaction blocks of different sizes), thereby obtaining sub-transaction blocks. Additional check blocks (redundant blocks) are generated according to the number of check blocks. For example, the erasure code algorithm can be used to add m (the number of check blocks) data (check blocks) to n (the number of sub-transaction blocks) original data (sub-transaction blocks).

[0069] The benefits of this setting are: the method of processing transaction data and generating data blocks based on the preset number of consensus nodes can significantly improve transaction processing efficiency, enhance data integrity and security, and improve scalability and resource utilization. These technical effects make this method have important value and broad application prospects in practical applications.

[0070] In this embodiment of the present application, the formula for determining the number of sub-transaction blocks and the number of check blocks is:

[0071] n = ɑf + 1;

[0072] Among them, n represents the preset number of consensus nodes, f represents the number of verification blocks, (ɑ-1)×(f+1) represents the number of sub-transaction blocks, and ɑ is a preset parameter.

[0073] Wherein, ɑ is a natural number, for example, it can be 2 or 3. In this embodiment, the value of ɑ is determined according to a preset consensus algorithm and needs to be a number greater than 1.

[0074] The consensus algorithm can be the raft algorithm or the pbft algorithm. The PBFT algorithm is a consensus algorithm used in distributed systems, which aims to ensure that the system can reach consensus when there are Byzantine failures (i.e. nodes may exhibit arbitrary erroneous behaviors, including malicious behaviors). The Raft algorithm is a consensus algorithm used for consistency management in distributed systems, which aims to enable multiple nodes to reach consensus in a distributed system. The Raft algorithm is mainly used for leader election, log replication and fault recovery in distributed systems.

[0075] When the consensus algorithm is the raft algorithm, ɑ is 2, n=2f+1, f represents the number of check blocks, and f+1 represents the number of sub-transaction blocks. For example, if the consensus algorithm is the raft algorithm, the number of consensus nodes is 3, and 2 data blocks Data1, Data2 and 1 check block Data3 are generated.

[0076] When the consensus algorithm is the pbft (Practical Byzantine Fault Tolerance) algorithm, ɑ is 3, n=3f+1, f represents the number of check blocks, and 2f+1 represents the number of sub-transaction blocks. For example, if the consensus algorithm is the raft algorithm, the number of consensus nodes is 4, and 3 sub-transaction blocks Data1, Data2, and Data3 are generated, and 1 check block Data4.

[0077] The benefit of this setting is that by reasonably determining the number of sub-transaction blocks and the number of check blocks, it can significantly improve fault tolerance, optimize resource allocation, improve consensus efficiency, enhance scalability, and improve data integrity and security. These technical effects make this method of great value and broad application prospects in practical applications.

[0078] In the embodiment of the present application, the transaction data is processed according to the number of sub-transaction blocks and the number of check blocks to obtain a data block of the transaction data, including:

[0079] According to the number of sub-transaction blocks, the number of check blocks, and the transaction data, based on the preset first erasure coding algorithm, the data content in the sub-transaction block and the data content in the check block are determined to obtain a data block of the transaction data.

[0080] Among them, the erasure code algorithm is a data protection technology used to provide fault tolerance during data storage and transmission. By splitting data into multiple blocks and adding redundant information, the erasure code can restore the original data when part of the data is lost or damaged. Erasure codes are widely used in distributed storage systems, cloud storage, data backup and communication systems.

[0081] An erasure coding algorithm is preset as the first erasure coding algorithm. The preset first erasure coding algorithm can be used to obtain sub-transaction blocks and check blocks according to transaction data, the number of sub-transaction blocks, and the number of check blocks. For example, the transaction data can be allocated to the sub-transaction blocks, and the sub-transaction blocks can be processed by the first erasure coding algorithm, and the processed data can be allocated to the check blocks. In this embodiment, the preset first erasure coding algorithm is not specifically limited.

[0082] There are three main types of applications of erasure coding technology in distributed storage systems: array erasure coding (RAID5

[0083] (redundant array of independent disks, redundant array of independent disks), RAID6 (redundant array of independent disks, redundant array of independent disks), RS (Reed-Solomon) erasure code and LDPC (Low Density Parity Check Code, low density parity check erasure code), the first erasure code algorithm can be any one of them.

[0084] When the first erasure code algorithm is a Reed-Solomon erasure code, this erasure code algorithm is the most suitable erasure code algorithm. By giving n sub-transaction blocks (Data blocks) D1, D2...Dn, and a positive integer m (i.e., the number of check blocks), RS generates m coding blocks (i.e., check blocks) C1, C2...Cm according to the n data blocks. Here, the number of sub-transaction blocks and the number of check blocks are determined by the type of consensus algorithm. For example, when the consensus algorithm is raft and the number of consensus nodes is 3, the first erasure code algorithm parameter is RS (2, 1), thereby generating 2 data blocks Data1, Data2, and 1 check block Data3.

[0085] The advantage of this setting is that by determining the number of sub-transaction blocks and verification blocks through the preset number of consensus nodes and consensus algorithms, and processing transaction data, the system's fault tolerance, security, performance, scalability and user experience can be significantly improved. This method has broad application prospects and technical advantages in blockchain and other distributed systems.

[0086] S103: Store the sub-transaction block and the verification block in a preset database on the blockchain.

[0087] The preset database includes a database for storing the data block itself and a database for storing the signature data of the data block.

[0088] When the preset database is used to store the data block itself, and the memory of the preset database is relatively large, large-volume data can be directly stored in the blockchain. The capacity of the preset database can be: 500GB, 1TB, 1PB.

[0089] One consensus node corresponds to one preset database, and each preset database stores one data block or verification block.

[0090] Among them, in the embodiment of the present application, the sub-transaction block and the verification block are stored in a preset database on the blockchain, including:

[0091] Determine a hash value of the sub-transaction block as a first hash value, and determine a hash value of the verification block as a second hash value;

[0092] Performing signature calculation on the first hash value to obtain first signature data, and performing signature calculation on the second hash value to obtain second signature data; wherein the first signature data represents the data content in the sub-transaction block, and the second signature data represents the data content in the verification block;

[0093] The sub-transaction block, the verification block, the first signature data, and the second signature data are stored in a preset database on the blockchain.

[0094] Among them, the hash value of each sub-transaction block and the hash value of each verification block are unique, and are different from the hash values ​​of any other sub-transaction blocks and any verification blocks.

[0095] When it contains two data blocks (i.e., sub-transaction blocks) Data1 and Data2 and one check block Data3, the first hash value hash1 of Data1, the first hash value hash2 of Data2, and the second hash value hash3 of Data3 are obtained. Sign hash1 and hash2 to obtain the first signatures sign1 and sign2. Sign hash3 to obtain the second signature sign3. Then save sign1, sign2, and sign3 to the database used to save data block signature data. Save Data1, Data2, and Data3 to the database used to save data block signature data.

[0096] The benefit of this setting is that by storing sub-transaction blocks and verification blocks in a preset database on the blockchain and verifying them using hash values ​​and signature data, it can significantly improve data integrity and immutability, enhance data security, improve transparency and traceability, optimize data verification and consensus processes, improve fault tolerance, simplify data management and maintenance, and improve scalability.

[0097] In this embodiment of the present application, determining the hash value of the sub-transaction block as the first hash value includes:

[0098] Perform hash calculation on the transaction data according to the preset hash algorithm to obtain the hash value of the transaction data;

[0099] Determine the first identification data of the sub-transaction block according to the position of the sub-transaction block in the transaction data;

[0100] According to the hash value of the transaction data and the first identification data, the hash value of the sub-transaction block is determined as the first hash value.

[0101] Among them, the preset hash algorithm can be MD5 (Message Digest Algorithm 5), SHA-1 (Secure Hash Algorithm 1), SHA-256 (Secure Hash Algorithm 256), and SHA-3 (Secure Hash Algorithm 3) algorithms.

[0102] The first identification data of each data block (i.e., sub-transaction block) is different, so that the first hash value of each data block is different. The first identification data can be a number, such as 1, 2, 3... The first identification data can also be different letters, such as a, b, c,... ab, ac... It can also be a mixture of numbers and letters, such as: a1, a2, a3...

[0103] The transaction data can be ABC, and the data blocks are cut into A, B, and C. A is at the first position of the transaction data, so the first identification data of data block A can be 1. B is at the second position of the transaction data, so the first identification data of data block B can be 2. C is at the second position of the transaction data, so the first identification data of data block C can be 3.

[0104] The transaction data is input into a preset hash algorithm to obtain a hash value of the transaction data. The first identification data of the data block is confirmed according to the different positions of the data block in the transaction data. Then, the first identification data and the hash value of the transaction data are concatenated to obtain the hash value of the data block as the first hash value.

[0105] The benefits of this setting are: by determining the hash value (first hash value) of the sub-transaction block, the integrity and immutability of the data can be significantly improved, data security can be enhanced, transparency and traceability can be improved, data verification and consensus processes can be optimized, fault tolerance can be improved, data management and maintenance can be simplified, and scalability can be improved.

[0106] In the embodiment of the present application, determining the hash value of the check block as the second hash value includes:

[0107] Allocating second identification data to the check block;

[0108] According to the hash value of the transaction data and the second identification data, the hash value of the verification block is determined as the second hash value.

[0109] Among them, the second identification data of each check block is different, so that the second hash value of each check block is different, and the second identification data can be a number, such as 1, 2, 3... The second identification data can also be different letters, a, b, c, ... ab, ac... It can also be a mixture of numbers and letters, for example: a1, a2, a3..., as long as any second hash value is different in the end, and any first hash value and any second hash value are different, for example, the first hash value can be a1, a2, and the second hash value is a3.

[0110] The advantage of this setting is that by determining the hash value (second hash value) of the check block, the integrity and immutability of the data can be significantly improved, data security can be enhanced, transparency and traceability can be improved, data verification and consensus processes can be optimized, fault tolerance can be improved, data management and maintenance can be simplified, and scalability can be improved.

[0111] In this embodiment of the present application, the method further includes:

[0112] Receive the transaction ID of the transaction to be queried, determine the number of sub-transaction blocks corresponding to the transaction to be queried, and determine the sub-transaction block corresponding to the transaction ID as the target sub-transaction block;

[0113] If the number of target sub-transaction blocks is equal to the number of sub-transaction blocks corresponding to the transaction to be queried, the target sub-transaction blocks are restored to the transaction data of the transaction to be queried according to the preset second erasure coding algorithm.

[0114] Among them, the number of sub-transaction blocks corresponding to the transaction to be queried represents the number of sub-transaction blocks for restoring transaction data, which is determined by the consensus algorithm. If the consensus algorithm is the pbft algorithm, the number of sub-transaction blocks corresponding to the transaction to be queried is f+1. If the consensus algorithm is the raft algorithm, the number of sub-transaction blocks corresponding to the transaction to be queried is 2f+1, where f is the number of check blocks.

[0115] The second erasure coding algorithm is used to restore the sub-data block to transaction data. When the number of target sub-transaction blocks is equal to the number of sub-transaction blocks corresponding to the transaction to be queried, the second erasure coding algorithm restores the target sub-transaction block to transaction data. When the number of target sub-transaction blocks is less than the number of sub-transaction blocks corresponding to the transaction to be queried, it is necessary to supplement the corresponding number of missing check blocks, and restore the transaction data through the target sub-transaction blocks and check blocks.

[0116] The first erasure code algorithm and the second erasure code algorithm can be an erasure code algorithm. Here, in order to distinguish the different functions of the same erasure code algorithm, they are named first and second. The first erasure code here is used to obtain sub-transaction blocks and check blocks based on transaction data. The second erasure code algorithm here is used to restore transaction data based on sub-transaction blocks, or sub-transaction blocks and check blocks. For example: when the consensus algorithm is raft and the number of consensus nodes is 3, the first erasure code algorithm parameter is RS (2, 1), and the transaction data is input to generate 2 data blocks Data1 and Data2, and 1 check block Data3. When the user needs to query the corresponding transaction data, it is determined that there are two target sub-data blocks, and the second erasure code algorithm parameter is RS (2, 1), and two target sub-data blocks are input to obtain the transaction data. Or when the user needs to query the corresponding transaction data, it is determined that there are two target sub-data blocks, then the second erasure code algorithm parameter is RS(2,1), and any target sub-data block and a target check block are input to obtain the transaction data. Any target sub-data block is any one of Data1 and Data2.

[0117] If the number of target sub-transaction blocks is equal to the number of sub-transaction blocks corresponding to the transaction to be queried, the target sub-transaction blocks are restored to transaction data through the second erasure coding algorithm.

[0118] The advantage of this setting is that by receiving the transaction ID of the transaction to be queried, determining the number of sub-transaction blocks corresponding to the transaction to be queried, and determining the sub-transaction block corresponding to the transaction ID (target sub-transaction block), and then using the preset second erasure code algorithm to restore the target sub-transaction block to the transaction data of the transaction to be queried, it can significantly improve data query efficiency, enhance data integrity and reliability, improve fault tolerance, optimize resource utilization, improve scalability, enhance data security, and simplify data management and maintenance.

[0119] In this embodiment of the present application, the method further includes:

[0120] If the number of target sub-transaction blocks is less than the number of sub-transaction blocks corresponding to the transaction to be queried, then determine the difference between the number of target sub-transaction blocks and the number of sub-transaction blocks corresponding to the transaction to be queried;

[0121] Obtain the check block corresponding to the transaction ID, and determine the target check block from the check blocks corresponding to the transaction ID according to the difference;

[0122] According to the preset second erasure coding algorithm, the target sub-transaction block and the target check block are restored to the transaction data of the transaction to be queried.

[0123] The difference represents the number of missing target sub-transaction blocks. When the target sub-transaction block is missing, it is necessary to obtain a check block corresponding to the number of difference.

[0124] If the number of check blocks corresponding to the difference cannot be obtained, then the blockchain has crashed and is unavailable.

[0125] The number of target sub-transaction blocks is less than the number of sub-transaction blocks corresponding to the transaction to be queried, indicating that the target sub-transaction blocks are damaged or missing. In this case, the transaction data cannot be directly obtained from the target sub-transaction blocks. Therefore, it is necessary to supplement the target check blocks corresponding to the difference, and then obtain the transaction data based on the target check blocks, the target sub-transaction blocks, and the second erasure algorithm.

[0126] The benefits of this setting are: by determining the difference and obtaining the corresponding check block when the number of target sub-transaction blocks is less than the number of sub-transaction blocks corresponding to the transaction to be queried, and then using the preset second erasure code algorithm to restore the target sub-transaction block and the target check block to the transaction data of the transaction to be queried, it can significantly improve data recovery capabilities, enhance fault tolerance, improve data integrity and reliability, optimize resource utilization, improve scalability, enhance data security, and simplify data management and maintenance.

[0127] The embodiment of the present application provides a data processing method based on blockchain, which no longer uses external storage to store large amounts of data, so the original data is stored in the blockchain, making the entire business more streamlined, reducing maintenance costs and difficulties, and at the same time, by setting a check block, it is allowed that a part of the target sub-data block fails, and the corresponding transaction data can also be queried, thereby improving the fault tolerance rate.

[0128] Figure 2 A flowchart of another data processing method based on blockchain provided in an embodiment of the present application. Figure 2 As shown, the execution subject of the method can be a server, and this embodiment does not make any special restrictions here. Figure 2 As shown, the method includes:

[0129] S201. Obtain transaction data.

[0130] Among them, the method for obtaining transaction data is:

[0131] When the client is uploading the transaction data, if the transaction data is greater than or equal to the preset threshold, the transaction data Tx is constructed as the MSG_BIGDATA_TRANSACTION type, and the transaction data is placed in the bigData field of Tx, and then the constructed Tx is sent to a consensus node in the blockchain network.

[0132] If the transaction data is less than the preset threshold, consensus is directly reached on the transaction data.

[0133] The preset threshold may be 500M.

[0134] S202. Use the Sha256 algorithm to calculate the bigData in Tx to obtain the hash of the transaction data.

[0135] S203. Determine the number of data blocks and the number of check blocks according to the current consensus algorithm type.

[0136] Among them, according to the current consensus algorithm type, the method for determining the number of data blocks and check blocks is:

[0137] If it is the raft algorithm, then n = 2f + 1, n is the total number of nodes, f is the number of fault-tolerant nodes of the consensus algorithm, which is also the number of check blocks, and f + 1 is the number of data blocks. If it is the pbft algorithm, then n = 3f + 1, n is the total number of nodes, f is the number of fault-tolerant nodes of the consensus algorithm, which is also the number of check blocks, and 2f + 1 is the number of data blocks.

[0138] The check block is the check data calculated by transaction data and erasure code algorithm, and is used to recover data when part of the data block is lost.

[0139] Data block: According to the number of data blocks, the transaction data is divided into multiple parts, each of which is a data block.

[0140] S204. Obtain data blocks and check blocks according to the number of data blocks, the number of check blocks, transaction data, and an erasure code algorithm.

[0141] Among them, according to the number of data blocks, the number of check blocks, transaction data, and the erasure code algorithm, the check block and the check block method are obtained as follows:

[0142] If the consensus algorithm is raft and the number of consensus nodes is 3, the erasure code algorithm parameters are RS(2,1) to generate 2 data blocks Data1 and Data2 and 1 check block Data3.

[0143] S205. Store the check block and the data block into the local large-volume data temporary database bigDataTempDB of the consensus node.

[0144] S206: Obtain the hash value of each data block and the hash value of each check block, and sign them using the node key.

[0145] The method of obtaining the hash value of each data block and the hash value of each check block and signing them with the node key is as follows:

[0146] The consensus node generates a hash value for each data block and a hash value for each check block based on the hash of the data block and the transaction data, and uses the key to sign the hash value of each data block and the hash value of each check block respectively. The signed hash value is then saved in the BigDataInfo field of Tx to form a new transaction NewTx, and consensus is reached on NewTx.

[0147] S207. Save each check block, data block, hash value of the completed signature of each data block, and hash value of the completed signature of each check block on the blockchain.

[0148] The method for saving each check block, data block, hash value of each data block with completed signature, and hash value of each check block with completed signature to the blockchain is:

[0149] After the consensus node completes the consensus on NewTx and writes the data into the local ledger, it determines the transaction type. If the transaction is of MSG_BIGDATA_TRANSACTION type, it parses the bigDataInfo field in the transaction, stores the hash value of the completed signature of each data block and the hash value of the completed signature of each verification block in the local data block information database (that is, the database on the blockchain used to store signatures), and writes the data block information that needs to be synchronized by this node into the data block synchronization table.

[0150] After the consensus node is started, it checks the local data block synchronization table at a certain time (configurable). If the data block synchronization table is not empty, it reads the data block records in the table in order and requests the corresponding data block from the target node recorded in the data block synchronization table.

[0151] After receiving the synchronization request, the target node verifies the request online, then reads the corresponding data block from the local large-volume temporary database BigDataTempDB and returns it to the requester, and deletes the corresponding data block in BigDataTempDB.

[0152] After receiving the response, the requesting node writes the data block into the local large-volume database BigDataDB and deletes the corresponding synchronization record in the data block synchronization table.

[0153] S208. The client sends a data query request transaction ID to the consensus node to query the original transaction data.

[0154] The client sends a data query request transaction ID to the consensus node to query the original transaction data as follows:

[0155] The node determines the number of data blocks corresponding to the transaction ID through the transaction ID. If it is the raft algorithm (or the data transaction corresponding to the transaction ID is the raft algorithm), then the number of data blocks is f+1. According to the transaction ID and the number of data blocks, f+1 target data blocks are confirmed.

[0156] If it is the pbft algorithm (or the pbft algorithm for data transactions corresponding to the transaction ID), the number of data blocks is 2f+1. According to the transaction ID and the number of data blocks, 2f+1 target data blocks are confirmed. If a node fails and cannot obtain enough data blocks, it polls other consensus nodes storing check blocks to request check blocks until the total number of data blocks and check blocks obtained reaches f+1 or 2f+1. Finally, the obtained data blocks and check blocks are restored to the original data through the erasure code module and returned to the client.

[0157] Another data processing method based on blockchain provided by the embodiment of the present application no longer uses external storage to store large-volume data, so the original data is stored in the blockchain, making the entire business more streamlined. It reduces the maintenance cost and difficulty, and uses erasure code technology to block large-volume data and generate check blocks. Each blockchain consensus node only stores one of the data blocks, which effectively reduces the storage consumption of blockchain by large-volume data. The number of check blocks is determined according to the type of consensus algorithm adopted by the blockchain, and the number of fault-tolerant nodes of the consensus algorithm is used as the number of check blocks generated. When the number of faulty nodes is not greater than the number of check blocks, the complete original data can still be restored based on the remaining data blocks and check blocks. Therefore, the fault tolerance of large-volume data is consistent with the fault tolerance of the consensus algorithm adopted, which greatly improves the risk resistance of large-volume data.

[0158] Figure 3 This is a structural example diagram of a data processing device based on blockchain provided in an embodiment of the present application. Figure 3 As shown, the blockchain-based data processing device 30 includes: an acquisition module 301, a obtaining module 302, and a storage module 303.

[0159] in:

[0160] An acquisition module 301 is used to acquire transaction data;

[0161] The obtaining module 302 is used to process the transaction data according to the preset number of consensus nodes to obtain a data block of the transaction data; wherein the data block includes a sub-transaction block and a check block, the sub-transaction block represents part of the transaction data, the check block is used to replace the sub-transaction block, and the sum of the number of the sub-transaction blocks and the check block is the preset number of consensus nodes;

[0162] The storage module 303 is used to store the sub-transaction block and the verification block in a preset database on the blockchain.

[0163] In one example, the module 302 is obtained, including:

[0164] A first determining unit, configured to determine the number of sub-transaction blocks and the number of verification blocks according to a preset number of consensus nodes and based on a preset consensus algorithm;

[0165] The first obtaining unit is used to process the transaction data according to the number of sub-transaction blocks and the number of check blocks to obtain a data block of the transaction data.

[0166] In a possible implementation, the determining unit is specifically configured to:

[0167] n = ɑf + 1;

[0168] Among them, n represents the preset number of consensus nodes, f represents the number of verification blocks, (ɑ-1)×(f+1) represents the number of sub-transaction blocks, and ɑ is a preset parameter.

[0169] In one example, the module 302 is obtained, including:

[0170] The second determination unit is used to determine the data content in the sub-transaction block and the data content in the check block according to the number of sub-transaction blocks, the number of check blocks, and the transaction data, based on a preset first erasure code algorithm, to obtain a data block of the transaction data.

[0171] In one example, the storage module 303 includes:

[0172] a third determining unit, configured to determine a hash value of the sub-transaction block as a first hash value, and to determine a hash value of the check block as a second hash value;

[0173] A second obtaining unit is used to perform signature calculation on the first hash value to obtain first signature data, and to perform signature calculation on the second hash value to obtain second signature data; wherein the first signature data represents the data content in the sub-transaction block, and the second signature data represents the data content in the verification block;

[0174] The storage unit is used to store the sub-transaction block, the verification block, the first signature data, and the second signature data in a preset database on the blockchain.

[0175] In a possible implementation manner, the third determining unit is specifically configured to:

[0176] Perform hash calculation on the transaction data according to the preset hash algorithm to obtain the hash value of the transaction data;

[0177] Determine the first identification data of the sub-transaction block according to the position of the sub-transaction block in the transaction data;

[0178] According to the hash value of the transaction data and the first identification data, the hash value of the sub-transaction block is determined as the first hash value.

[0179] In a possible implementation manner, the third determining unit is specifically configured to:

[0180] Allocating second identification data to the check block;

[0181] According to the hash value of the transaction data and the second identification data, the hash value of the verification block is determined as the second hash value.

[0182] In one example, the storage module 303 includes:

[0183] A fourth determining unit, configured to receive a transaction ID of the transaction to be queried, determine the number of sub-transaction blocks corresponding to the transaction to be queried, and determine the sub-transaction block corresponding to the transaction ID as a target sub-transaction block;

[0184] The restoration unit is used to restore the target sub-transaction blocks to the transaction data of the transaction to be queried according to a preset second erasure coding algorithm if the number of target sub-transaction blocks is equal to the number of sub-transaction blocks corresponding to the transaction to be queried.

[0185] In a possible implementation, the restoration unit is specifically configured to:

[0186] If the number of target sub-transaction blocks is less than the number of sub-transaction blocks corresponding to the transaction to be queried, then determine the difference between the number of target sub-transaction blocks and the number of sub-transaction blocks corresponding to the transaction to be queried;

[0187] Obtain the check block corresponding to the transaction ID, and determine the target check block from the check blocks corresponding to the transaction ID according to the difference;

[0188] According to the preset second erasure coding algorithm, the target sub-transaction block and the target check block are restored to the transaction data of the transaction to be queried.

[0189] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 40 includes:

[0190] The electronic device 40 may include a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a communication component 403 and other components. The processor 401 , the memory 402 and the communication component 403 are connected via a bus 404 .

[0191] In the specific implementation process, at least one processor 401 executes the computer execution instructions stored in the memory 402, so that at least one processor 401 executes the above blockchain-based data processing method.

[0192] The specific implementation process of the processor 401 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0193] In the above Figure 4 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0194] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0195] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0196] In some embodiments, a computer program product is also proposed, including a computer program or instructions, which, when executed by a processor, implement the steps in any of the above-mentioned blockchain-based data processing methods.

[0197] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0198] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0199] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple instructions are stored, and the instructions can be loaded by a processor to execute the steps of any blockchain-based data processing method provided in the embodiment of the present application.

[0200] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0201] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program comprises computer instructions stored in a computer-readable storage medium.

[0202] Since the instructions stored in the storage medium can execute the steps in any one of the blockchain-based data processing methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the blockchain-based data processing methods provided in the embodiments of the present application can be achieved. For details, please refer to the previous embodiments and will not be repeated here.

[0203] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0204] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data processing method based on blockchain, characterized in that: The method comprises: Get transaction data; According to the preset number of consensus nodes, the transaction data is processed to obtain a data block of the transaction data; wherein the data block includes a sub-transaction block and a check block, the sub-transaction block represents part of the transaction data, the check block is used to replace the sub-transaction block, and the sum of the number of the sub-transaction blocks and the check block is the preset number of consensus nodes; The sub-transaction block and the verification block are stored in a preset database on the blockchain.

2. The method according to claim 1, characterized in that The processing of the transaction data according to the preset number of consensus nodes to obtain a data block of the transaction data includes: According to the preset number of consensus nodes and based on a preset consensus algorithm, determine the number of sub-transaction blocks and the number of verification blocks; The transaction data is processed according to the number of the sub-transaction blocks and the number of the check blocks to obtain a data block of the transaction data.

3. The method according to claim 2, characterized in that The formula for determining the number of the sub-transaction blocks and the number of the verification blocks is: n = ɑf + 1; Among them, n represents the preset number of consensus nodes, f represents the number of verification blocks, (ɑ-1)×(f+1) represents the number of sub-transaction blocks, and ɑ is a preset parameter.

4. The method according to claim 1, characterized in that The processing of the transaction data according to the number of the sub-transaction blocks and the number of the check blocks to obtain a data block of the transaction data includes: According to the number of the sub-transaction blocks, the number of the check blocks, and the transaction data, based on a preset first erasure code algorithm, the data content in the sub-transaction blocks and the data content in the check blocks are determined to obtain a data block of the transaction data.

5. The method according to claim 1, characterized in that The storing the sub-transaction block and the verification block in a preset database on the blockchain includes: Determine the hash value of the sub-transaction block as a first hash value, and determine the hash value of the verification block as a second hash value; Performing signature calculation on the first hash value to obtain first signature data, and performing signature calculation on the second hash value to obtain second signature data; wherein the first signature data represents the data content in the sub-transaction block, and the second signature data represents the data content in the verification block; The sub-transaction block, the verification block, the first signature data, and the second signature data are stored in a preset database on the blockchain.

6. The method according to claim 5, characterized in that Determining the hash value of the sub-transaction block as a first hash value includes: Performing hash calculation on the transaction data according to a preset hash algorithm to obtain a hash value of the transaction data; Determining first identification data of the sub-transaction block according to a position of the sub-transaction block in the transaction data; According to the hash value of the transaction data and the first identification data, the hash value of the sub-transaction block is determined as the first hash value.

7. The method according to claim 6, characterized in that The step of determining the hash value of the check block as a second hash value includes: Allocating second identification data to the check block; According to the hash value of the transaction data and the second identification data, the hash value of the verification block is determined as the second hash value.

8. The method according to claim 1, characterized in that The method further comprises: Receive a transaction ID of a transaction to be queried, determine the number of sub-transaction blocks corresponding to the transaction to be queried, and determine the sub-transaction block corresponding to the transaction ID as a target sub-transaction block; If the number of the target sub-transaction blocks is equal to the number of sub-transaction blocks corresponding to the transaction to be queried, the target sub-transaction blocks are restored to the transaction data of the transaction to be queried according to a preset second erasure coding algorithm.

9. The method according to claim 8, characterized in that The method further comprises: If the number of the target sub-transaction blocks is less than the number of sub-transaction blocks corresponding to the transaction to be queried, determining the difference between the number of the target sub-transaction blocks and the number of sub-transaction blocks corresponding to the transaction to be queried; Obtain a check block corresponding to the transaction ID, and determine a target check block from the check blocks corresponding to the transaction ID according to the difference; According to a preset second erasure coding algorithm, the target sub-transaction block and the target check block are restored to the transaction data of the transaction to be queried.

10. A data processing device based on blockchain, characterized in that: include: Acquisition module, used to obtain transaction data; An obtaining module is used to process the transaction data according to a preset number of consensus nodes to obtain a data block of the transaction data; wherein the data block includes a sub-transaction block and a check block, the sub-transaction block represents part of the transaction data, the check block is used to replace the sub-transaction block, and the sum of the number of the sub-transaction blocks and the check block is the preset number of consensus nodes; A storage module is used to store the sub-transaction block and the verification block in a preset database on the blockchain.

11. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

Images