A blockchain data distributed storage method, device and storage medium

Abstract

The present invention belongs to the field of computer technology, and in particular relates to a distributed storage method, device, and storage medium for blockchain data. Distributed storage and a dynamic balance backup strategy eliminate the need for each node to store all block data, thereby improving the utilization efficiency of nodes in the blockchain network, reducing the node's hard disk requirements, and saving node deployment costs. Furthermore, the method can flexibly respond to node downtime and exit and new node addition. Due to periodic statistical backup, the backup of archived data is in a dynamic balance, so that the downtime of existing nodes will not affect the storage of original archived data, and the addition of new nodes will not affect the location query of original data. Furthermore, data can be quickly acquired, ensuring the stability of the blockchain.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a distributed storage method, device and storage medium for blockchain data. Background Technology

[0002] Currently, blockchain data storage involves each node maintaining an independent copy of the data, and all nodes should maintain consistent data. The advantage of each node maintaining an independent copy is that data verification does not rely on the reliability of the network or other nodes. However, the disadvantages are also obvious. Blockchain nodes can continue to grow, resulting in thousands of data backups. More importantly, block data represents all historical data since the blockchain's inception. As the blockchain network continues to operate, the scale of blockchain data will continue to grow, and the amount of data that needs to be stored will also increase. Ordinary nodes would need a large amount of disk space just to store this data, and when the data volume reaches a certain level, it becomes difficult for ordinary nodes to store such a large volume of data.

[0003] Many existing blockchains currently employ models such as lightweight nodes. However, when these nodes verify data, they rely on the reliability of the network and other nodes, increasing the interaction between nodes, consuming network resources, and making the reliability of the data dependent on other nodes. Summary of the Invention

[0004] This invention provides a distributed storage method, device, and storage medium for blockchain data. By archiving and packaging historical block data and storing it in shards, the number of backups of historical block data in the blockchain network is reduced, the hard drive requirements of a single node are alleviated, and node deployment costs are saved.

[0005] To solve or partially solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a distributed storage method for blockchain data, applicable to blockchain nodes, the method comprising:

[0007] Check the number of blocks stored locally on the node to determine if the number of blocks has exceeded the safe rollback depth. If so, continue to determine if the excess number exceeds the first number. If so, package the blocks stored locally according to the preset packaging rules to generate the first archive data.

[0008] The hash of the first archived data is obtained by performing a hash operation on the first archived data.

[0009] The node hashes of each node stored in the node's local routing table are obtained, and the logical distance between the obtained node hashes and the hash of the first archived data is calculated. Based on the logical distance to the hash of the first archived data, a second number of nodes corresponding to the node hashes are selected as the first node in order of proximity. The node hash is obtained by hashing the unique identifier of each node. The unique identifier is either the node ID or the node address. The local routing table is used to store the node information of the nodes in the blockchain. The logical distance is the difference between two hash values, and the one with the smaller difference is defined as the closer one.

[0010] If the current node belongs to the first node, the first archived data is stored with a validity period of a first duration, and the hash of the first archived data is stored in the node's archived data-archived data hash mapping table and associated with the first archived data. The first archived data is then sent to the remaining first nodes to store the first archived data. Otherwise, the first archived data is deleted after the first duration, and the first archived data is sent to the remaining first nodes to store the second archived data. The archived data-archived data hash mapping table is used to store the archived data hash corresponding to each archived data.

[0011] When the backup verification period arrives, all archived data stored in the node is traversed. According to the archived data-archived data hash mapping table, the hash of all archived data stored in the node is sent to the statistical script of the blockchain network. After receiving the statistical end notification sent by the statistical script, the first backup quantity is determined for each archived data stored in the node according to the current backup quantity table of archived data in the whole network published by the statistical script, the preset expected backup quantity in the node, and the archived data backup rules.

[0012] Based on the local routing table and the archive data-archive data hash mapping table, determine the nearest node for each archive data stored in the node, corresponding to the first number of backups, and send the corresponding archive data backup notification.

[0013] In response to the second archive data backup notification sent by the second node, the second archive data backup notification includes the second archive data hash corresponding to the second archive data. It is determined whether the hash corresponding to each archive data stored locally by the node is consistent with the second archive data hash. If there is no consistency, a first backup request is sent to the second node. If there is a consistency, the validity period of the second archive data is extended by a first duration.

[0014] Receive a second backup request sent by a third node, the second backup request including a third archive data hash, obtain the third archive data locally according to the third archive data hash and send it to the third node to store the third archive data;

[0015] Receive the fourth archived data sent by the fourth node, calculate the corresponding hash of the fourth archived data, store the fourth archived data and set the validity period to a first duration, and store the hash of the fourth archived data in the archived data-archived data hash mapping table corresponding to the fourth archived data.

[0016] As a preferred method, the first backup quantity for each archived data stored in the node is determined based on the publicly available table of current backup quantities of archived data across the entire network, the preset expected backup quantity within the node, and the archived data backup rules.

[0017] The expected backup quantity specifically includes the default backup quantity, the minimum backup quantity, and the maximum backup quantity; the archived data backup rules include:

[0018] The system queries the current backup count of the archived data from the table of current backup counts. If no backup count is found, the default backup count is used as the first backup count. If the current backup count is less than the minimum backup count, the minimum backup count is used as the first backup count. If the current backup count is greater than the maximum backup count, the maximum backup count is used as the first backup count. Otherwise, the current backup count is used as the first backup count.

[0019] Preferably, the packaging rule includes selecting the block content of a first number of blocks with the smallest block height from the node's local storage and packaging them.

[0020] In a second aspect, a computer device is provided, comprising one or more processors;

[0021] Memory, used to store one or more programs.

[0022] When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect above.

[0023] Thirdly, a storage medium storing a computer program is provided, which, when executed by a processor, implements the method described in the first aspect above.

[0024] The beneficial effects of this invention are that the distributed storage method for blockchain data archives and packages blocks before the safe rollback interval through distributed storage, and then stores them in shards in the blockchain network. This eliminates the need for each node to store all block data, improving the utilization efficiency of nodes in the blockchain network, reducing the hard drive requirements of nodes, and saving node deployment costs. At the same time, the method of locating storage nodes by logical distance and periodic statistical backups can flexibly respond to node failures and the addition of new nodes. Due to the periodic statistical backups, the backup of archived data is in a dynamic balance, so that the failure of existing nodes will not affect the storage of the original archived data, and the addition of new nodes will not affect the location and query of the original data, and data can be quickly obtained, ensuring the stability of the blockchain. Attached Figure Description

[0025] Figure 1 A flowchart illustrating a distributed storage method for blockchain data according to an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0027] The present invention will be further described clearly and in detail below with reference to specific embodiments and accompanying drawings. Those skilled in the art will be able to implement the present invention based on these descriptions. Furthermore, the embodiments of the present invention described below are generally only some embodiments of the present invention, and not all embodiments. Therefore, all other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort should fall within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. Unless otherwise specified, the methods briefly described in the embodiments of the present invention are methods mastered by those skilled in the art.

[0028] Figure 1 This is a flowchart of a blockchain data distributed storage method provided in an embodiment of the present invention.

[0029] As an embodiment of the present invention, a distributed storage method for blockchain data is provided, such as... Figure 1 As shown, applicable to blockchain nodes, the method includes:

[0030] Check the number of blocks stored locally on the node to determine if the number of blocks has exceeded the safe rollback depth. If so, continue to determine if the excess number exceeds the first number. If so, package the blocks stored locally according to the preset packaging rules to generate the first archive data.

[0031] The hash of the first archived data is obtained by performing a hash operation on the first archived data.

[0032] The node hashes of each node stored in the node's local routing table are obtained, and the logical distance between the obtained node hashes and the hash of the first archived data is calculated. Based on the logical distance to the hash of the first archived data, a second number of nodes corresponding to the node hashes are selected as the first node in order of proximity. The node hash is obtained by hashing the unique identifier of each node. The unique identifier is either the node ID or the node address. The local routing table is used to store the node information of the nodes in the blockchain. The logical distance is the difference between two hash values, and the one with the smaller difference is defined as the closer one.

[0033] If the current node belongs to the first node, the first archived data is stored with a validity period of a first duration, and the hash of the first archived data is stored in the node's archived data-archived data hash mapping table and associated with the first archived data. The first archived data is then sent to the remaining first nodes to store the first archived data. Otherwise, the first archived data is deleted after the first duration, and the first archived data is sent to the remaining first nodes to store the second archived data. The archived data-archived data hash mapping table is used to store the archived data hash corresponding to each archived data.

[0034] When the backup verification period arrives, all archived data stored in the node is traversed. According to the archived data-archived data hash mapping table, the hash of all archived data stored in the node is sent to the statistical script of the blockchain network. After receiving the statistical end notification sent by the statistical script, the first backup quantity is determined for each archived data stored in the node according to the current backup quantity table of archived data in the whole network published by the statistical script, the preset expected backup quantity in the node, and the archived data backup rules.

[0035] Based on the local routing table and the archive data-archive data hash mapping table, determine the nearest node for each archive data stored in the node, corresponding to the first number of backups, and send the corresponding archive data backup notification.

[0036] In response to the second archive data backup notification sent by the second node, the second archive data backup notification includes the second archive data hash corresponding to the second archive data. It is determined whether the hash corresponding to each archive data stored locally by the node is consistent with the second archive data hash. If there is no consistency, a first backup request is sent to the second node. If there is a consistency, the validity period of the second archive data is extended by a first duration.

[0037] Receive a second backup request sent by a third node, the second backup request including a third archive data hash, obtain the third archive data locally according to the third archive data hash and send it to the third node to store the third archive data;

[0038] Receive the fourth archived data sent by the fourth node, calculate the corresponding hash of the fourth archived data, store the fourth archived data and set the validity period to a first duration, and store the hash of the fourth archived data in the archived data-archived data hash mapping table corresponding to the fourth archived data.

[0039] Specifically, based on the publicly available table of current backup quantities of archived data across the entire network, the preset expected backup quantities within the node, and the archived data backup rules, the first backup quantity for each archived data stored in the node is determined, including:

[0040] The expected backup quantity specifically includes the default backup quantity, the minimum backup quantity, and the maximum backup quantity; the archived data backup rules include:

[0041] The system queries the current backup count of the archived data from the table of current backup counts. If no backup count is found, the default backup count is used as the first backup count. If the current backup count is less than the minimum backup count, the minimum backup count is used as the first backup count. If the current backup count is greater than the maximum backup count, the maximum backup count is used as the first backup count. Otherwise, the current backup count is used as the first backup count.

[0042] Specifically, the packaging rule includes selecting the block content of the first number of blocks with the smallest block height from the node's local storage and packaging them.

[0043] More specifically, assuming the current block height is 15160, the safe rollback depth is 1000, the first quantity is set to 100, the last archived data is 13901-14000, the first duration is set to 2 hours, the second duration is set to one day, the backup cycle is consistent with the first duration, the default backup quantity, minimum backup quantity, and maximum backup quantity are 20, 10, and 30 respectively, and the second quantity is usually consistent with the default backup quantity. The purpose of setting the safe rollback depth and packing rules is to retain a certain number of the latest blocks without archiving them. Here, it ensures that at least the latest 1000 blocks are not packed. The first quantity sets the number of blocks to be archived each time, here 100 blocks are archived each time. At this point, the unarchived data exceeds 1100, meeting the condition for archiving. According to the packing rules, the node should pack the earliest 100 unarchived blocks, i.e., blocks 14001-1. Archive 4100 is packaged and denoted as chunk1 (14001-14100). The hash of chunk1 is calculated to obtain hash(chunk1). The node finds the 20 logically closest nodes based on its local routing table and hash(chunk1), assuming they are N1-N20. Then the node checks whether it belongs to N1-N20. For example, if the current node is N29, it does not belong to the 20 closest nodes. So it sends chunk1 to N1-N20 and deletes chunk1 itself after one day. If it does, it stores chunk1 (14001-14100) with a validity period of 2 hours. Then it stores hash(chunk1) in the archive data-archive data hash mapping table and associates it with the corresponding chunk1. Then it sends chunk1 to the remaining 19 nodes. It does not need to delete chunk1 itself. In fact, the node hash is fixed because the node hash is obtained by hashing the unique identifier of each node. The unique identifier is either the node ID or the node address. Therefore, the position of the node can be considered fixed. The principle of this application is to store the archived data whose hash falls between two node hashes into these two nodes and several nodes near the two nodes. "Nearby" means that the values ​​of the node hashes are close. When querying, the closest node is found through the archived data hash. If it is not found, the scope is expanded to find other nodes.

[0044] Two hours after the last backup, the entire blockchain network performs another backup. Nodes traverse all stored archived data, such as chunk2-chunk10. Then, based on the archived data-archived data hash mapping table, each node sends the hashes of all stored archived data and hash(chunk2)-hash(chunk10) to the blockchain network's statistics script. After receiving the statistics script's notification of the end of statistics, each node determines the number of backups for chunk2-chunk10. Then, based on the backup number of each archived data, it sends backup notifications to the nodes with the nearest backup number. Taking chunk2 as an example, the node first uses hash(chunk2) to query the publicly available table of current backup numbers of archived data across the entire network, finds the current backup number of chunk2 (assuming 5 copies), which is less than the minimum backup number of 10 copies. Therefore, according to the archived data backup rules, the backup number of chunk2 should be 10. The node selects the 10 nodes closest to hash(chunk2) for backup. Similarly, it also needs to determine whether the current node belongs to the 10 closest nodes. The specific process is similar to the storage process described above and will not be repeated here. Similarly, if the current backup count of chunk2 is 15, then 15 nodes will be selected for backup; if it is 40, then 30 nodes will be selected for backup; if chunk2 is not found, it means the data has just been stored and has not yet been backed up, so the default backup count of 20 will be used.

[0045] At the same time, the node will also receive backup notifications from other nodes. At this time, the node needs to check whether there is archived data that needs to be backed up in its own node. If there is, the node will extend the validity period of the data. If not, the node will send a backup request to the node that sent the backup notification to obtain the archived data and then set the corresponding validity period.

[0046] The advantage of this embodiment lies in the dynamic nature of the archived data stored in the nodes. Archived data expires without a backup notification, ensuring a dynamic balance of archived data across the blockchain network. The number of backups remains within a set range, allowing for flexible responses to node failures and new node additions. Due to periodic statistical backups, the archived data is in a dynamic balance, preventing the failure of existing nodes from affecting the storage of existing archived data, and ensuring that the addition of new nodes does not hinder the location and retrieval of existing data, enabling rapid data retrieval and guaranteeing blockchain stability. Furthermore, the distributed storage method eliminates the need for each node to store all block data, improving node utilization efficiency in the blockchain network and reducing hard drive requirements, thus saving on node deployment costs.

[0047] Figure 2 This is a schematic diagram of the structure of a device provided in an embodiment of the present invention.

[0048] like Figure 2 As shown, in another embodiment of the present invention, a computer device 100 is provided, including one or more central processing units (CPUs) 101, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 102 or programs loaded from storage portion 108 into random access memory (RAM) 103. The RAM 103 also stores various programs and data required for the operation of the device 100. The CPU 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0049] The following components are connected to I / O interface 105: an input section 106 including a keyboard, mouse, etc.; an output section 107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 108 including a hard disk, etc.; and a communication section 109 including a network interface card such as a LAN card, modem, etc. The communication section 109 performs communication processing via a network such as the Internet. Driver 110 is also connected to I / O interface 105 as needed. Removable media 111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on driver 110 as needed so that computer programs read from them can be installed into storage section 108 as needed.

[0050] Specifically, according to the embodiments disclosed in this application, the method described in Embodiment 1 above can be implemented as a computer software program. For example, embodiments disclosed in this application include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the methods described in any of the embodiments above. In such embodiments, the computer program can be downloaded and installed from a network via communication section 109, and / or installed from removable medium 111.

[0051] In another aspect, this application also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the apparatus of the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs that are used by one or more processors to perform the methods described in this application.

[0052] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or operation, or by a combination of dedicated hardware and computer instructions.

[0053] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be located in a processor; for example, each unit can be a software program located in a computer or mobile smart device, or a separately configured hardware device. The names of these units or modules do not, in some cases, constitute a limitation on the unit or module itself.

[0054] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the concept of this application. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A distributed storage method for blockchain data, characterized in that, Applicable to blockchain nodes, the method includes: Check the number of blocks stored locally on the node to determine if the number of blocks has exceeded the safe rollback depth. If so, continue to determine if the excess number exceeds the first number. If so, package the blocks stored locally according to the preset packaging rules to generate the first archive data. The hash of the first archived data is obtained by performing a hash operation on the first archived data. The node hashes of each node stored in the node's local routing table are obtained, and the logical distance between the obtained node hashes and the hash of the first archived data is calculated. Based on the logical distance to the hash of the first archived data, a second number of nodes corresponding to the node hashes are selected as the first node in order of proximity. The node hash is obtained by hashing the unique identifier of each node. The unique identifier is either the node ID or the node address. The local routing table is used to store the node information of the nodes in the blockchain. The logical distance is the difference between two hash values, and the one with the smaller difference is defined as the closer one. If the current node belongs to the first node, the first archived data is stored with a validity period of a first duration, and the hash of the first archived data is stored in the node's archived data-archived data hash mapping table and associated with the first archived data. The first archived data is then sent to the remaining first nodes to store the first archived data. Otherwise, the first archived data is deleted after the first duration, and the first archived data is sent to the remaining first nodes to store the second archived data. The archived data-archived data hash mapping table is used to store the archived data hash corresponding to each archived data. When the backup verification period arrives, all archived data stored in the node is traversed. According to the archived data-archived data hash mapping table, the hash of all archived data stored in the node is sent to the statistical script of the blockchain network. After receiving the statistical end notification sent by the statistical script, the first backup quantity is determined for each archived data stored in the node according to the current backup quantity table of archived data in the whole network published by the statistical script, the preset expected backup quantity in the node, and the archived data backup rules. Based on the local routing table and the archive data-archive data hash mapping table, determine the nearest node for each archive data stored in the node, corresponding to the first number of backups, and send the corresponding archive data backup notification. In response to the second archive data backup notification sent by the second node, the second archive data backup notification includes the second archive data hash corresponding to the second archive data. It is determined whether the hash corresponding to each archive data stored locally by the node is consistent with the second archive data hash. If there is no consistency, a first backup request is sent to the second node. If there is a consistency, the validity period of the second archive data is extended by a first duration. Receive a second backup request sent by a third node, the second backup request including a third archive data hash, obtain the third archive data locally according to the third archive data hash and send it to the third node to store the third archive data; Receive the fourth archived data sent by the fourth node, calculate the corresponding hash of the fourth archived data, store the fourth archived data and set the validity period to a first duration, and store the hash of the fourth archived data in the archived data-archived data hash mapping table corresponding to the fourth archived data.

2. The blockchain data distributed storage method according to claim 1, characterized in that, Based on the publicly available table of current backup quantities of archived data across the entire network, the preset expected backup quantities within the node, and the archived data backup rules, the first backup quantity for each archived data stored in the node is determined, including: The expected backup quantity specifically includes the default backup quantity, the minimum backup quantity, and the maximum backup quantity; the archived data backup rules include: The system queries the current backup count of the archived data from the table of current backup counts. If no backup count is found, the default backup count is used as the first backup count. If the current backup count is less than the minimum backup count, the minimum backup count is used as the first backup count. If the current backup count is greater than the maximum backup count, the maximum backup count is used as the first backup count. Otherwise, the current backup count is used as the first backup count.

3. A blockchain data distributed storage method according to claim 1 or 2, characterized in that, The packaging rules include selecting the block content of the first number of blocks with the smallest block height from the node's local storage and packaging them.

4. A computer device, characterized in that, One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in claim 1, 2 or 3.

5. A storage medium storing a computer program, characterized in that, When the program is executed by the processor, it implements the method as described in claim 1, 2, or 3.

Images