Blockchain storage efficiency optimization method
Through distributed hash tables and distributed storage technology, the problem of low blockchain storage efficiency and insufficient reliability is solved, efficient storage and data privacy protection is achieved, and node storage is significantly saved Space and improve the overall reliability of blockchain.
Patent Information
- Application Number
- PCT/CN2023/136353
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-25
- Filing Date
- 2023-12-05
- Publication Date
- 2024-11-28
AI Technical Summary
When the existing blockchain storage solution expands in the scale of nodes, the storage efficiency is low and the reliability is insufficient. In particular, only a single or a few copies exist when the IPFS node first uploads data, resulting in a reduction in blockchain reliability.
Using distributed hash table and distributed storage technology, block data is divided into multiple data slices and backed up in the entire network node. Each data slice is saved on multiple nodes and passed the distributed hash table. Record the node ID and data location to ensure data privacy and high reliability.
Without affecting the security and performance of blockchain, the storage efficiency of blockchain nodes is significantly improved, reducing the total storage volume by about 50%, and improving the reliability and security of blockchain.
Smart Images

Figure CN2023136353_28112024_PF_FP_ABST
Abstract
Description
A blockchain storage efficiency optimization method Technical Field
[0001] The present invention relates to the field of blockchain technology, and in particular to a method for optimizing blockchain storage efficiency. Background Art
[0002] Blockchain boasts characteristics such as data immutability, fairness, transparency, and decentralization, making it widely used in distributed applications such as digital assets, electronic evidence storage, and electronic traceability. To ensure these characteristics, all participating blockchain nodes must fully store all block data on the chain. A large number of nodes places extremely high demands on total storage costs. Furthermore, each transaction generated on the blockchain includes not only the submitted business information but also the complete transaction information, consensus signatures, block headers, and other accompanying data. This results in the actual size of each transaction data being significantly larger than the business data it carries. Therefore, as the number of blockchain nodes continues to expand, optimizing the storage structure across all nodes and improving overall storage efficiency becomes a pressing issue. Technical issues
[0003] CN113157693A discloses a blockchain storage optimization solution for digital currencies. This solution stores block data in IPFS, requiring blockchain nodes to store only IPFS index keywords. This approach reduces node storage consumption, but direct use of IPFS for storage has certain limitations. When an IPFS node initially uploads data, it only stores the data locally on that node and synchronizes it across the entire network until another node requests the data. This results in a long period of time when a block is first generated, with only a single or a few copies remaining, significantly reducing the reliability of the blockchain. Technical Solutions
[0004] To overcome the above problems or at least partially solve them, embodiments of the present invention provide a blockchain storage efficiency optimization method that significantly improves the overall storage efficiency of all blockchain nodes without affecting the security, reliability, and original performance of the blockchain.
[0005] The embodiment of the present invention is achieved as follows:
[0006] In a first aspect, an embodiment of the present invention provides a method for optimizing blockchain storage efficiency, comprising the following steps:
[0007] When a new blockchain node joins the blockchain, it generates a P2P communication certificate for the node. After the proposal to join passes consensus, the node certificate hash value is used as the ID, and the node ID and connection address are broadcast and saved in the distributed hash table of all nodes in the network.
[0008] When a transaction occurs on the chain and a block needs to be produced, the node responsible for packaging the block first divides the complete block data into multiple data slices, calculates the hash value of the overall block data and the hash value of each data slice, and then generates streamlined block data;
[0009] After the block data is packaged and generated, the node responsible for packaging the block will broadcast the simplified block data to other nodes in the entire network, and each node will save the simplified block data;
[0010] After the streamlined block data is successfully broadcast, the node responsible for packaging the block selects a number of nodes equal to the number of nodes in the entire network / 2+1 for each data slice according to the preset node selection rules, and sends each data slice to the distributed storage module of the selected node for storage;
[0011] After the data slices are successfully distributed, the node responsible for packaging and producing blocks will broadcast the distribution results to the entire network.
[0012] To address the challenges of existing technologies, this invention adds distributed storage and distributed hash table technology to existing blockchain storage. Distributed storage is used to store slices of the original block data that each node needs to store, while the distributed hash table stores the ID and address of each node in the blockchain, as well as the node where the data resides in the distributed storage. Block data is stored in distributed storage as slices, making it impossible to access the original block data through a single slice, effectively protecting data privacy. Each block data is backed up according to the number of nodes in the entire network / 2 + 1. This reduces the total storage capacity by nearly 50%, while ensuring that at least half of the nodes are honest.
[0013] The present invention uses a combination of distributed hash tables and distributed storage to replace the original single local storage mode of the blockchain. Without affecting the security, reliability, and original performance of the blockchain, it greatly improves the overall storage efficiency of all nodes in the blockchain; at the same time, it greatly saves node storage space.
[0014] Based on the first aspect, in some embodiments of the present invention, the blockchain storage efficiency optimization method further includes the following steps:
[0015] When a node needs to read block data, it searches for the corresponding block data based on the hash value of the block.
[0016] Based on the first aspect, in some embodiments of the present invention, the method for searching for corresponding block data according to the hash value of the block includes the following steps:
[0017] Query the simplified data of the block in the node storage and obtain the corresponding hash list of the included data slices;
[0018] For each data slice hash, first search in the local distributed storage. If the query is found, the corresponding data is directly returned. If not found, the node ID that owns the slice is searched in the distributed hash table. Then, the connection address of the node is searched in the distributed hash table. Finally, a data query is initiated to the node to obtain the original data of the slice.
[0019] The original data of all slices are spliced into complete block data.
[0020] Based on the first aspect, in some embodiments of the present invention, the method in which the node responsible for packaging a block first divides the complete block data into a plurality of data slices includes the following steps:
[0021] The node responsible for packaging the block first divides the complete block data into multiple data slices in the format of a byte array according to the preset slice size rules.
[0022] Based on the first aspect, in some embodiments of the present invention, the above-mentioned simplified block data includes block height, block hash value, preceding block hash value, block timestamp and data slice hash list.
[0023] In a second aspect, an embodiment of the present application provides an electronic device comprising a memory for storing one or more programs and a processor. When the one or more programs are executed by the processor, the method according to any one of the first aspects is implemented.
[0024] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method according to any one of the above-mentioned first aspects is implemented. Beneficial effects
[0025] The embodiments of the present invention have at least the following advantages or beneficial effects:
[0026] The embodiments of the present invention provide a method for optimizing blockchain storage efficiency. This method uses a combination of distributed hash tables and distributed storage to replace the original single local storage mode of the blockchain. Without affecting the security, reliability, and original performance of the blockchain, it greatly improves the overall storage efficiency of all nodes in the blockchain; at the same time, it greatly saves node storage space. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.
[0028] FIG1 is a flowchart of a method for optimizing blockchain storage efficiency according to an embodiment of the present invention;
[0029] FIG2 is a schematic diagram of a detailed flow chart of a method for optimizing blockchain storage efficiency according to an embodiment of the present invention;
[0030] FIG3 is a schematic diagram of blockchain node storage according to an embodiment of the present invention;
[0031] FIG4 is a structural block diagram of an electronic device provided by an embodiment of the present invention.
[0032] Description of the accompanying drawings: 101, memory; 102, processor; 103, communication interface. Modes for Carrying Out the Invention
[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.
[0034] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.
[0035] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.
[0036] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.
[0037] In the description of the embodiments of the present invention, "a plurality of" means at least two. Example:
[0038] As shown in FIG1 , in a first aspect, an embodiment of the present invention provides a method for optimizing blockchain storage efficiency, comprising the following steps:
[0039] S1. When a new blockchain node joins the blockchain, it generates a P2P communication certificate for the node. After the proposal to join passes consensus, the node certificate hash value is used as the ID, and the node ID and connection address are broadcast and saved in the distributed hash table of all nodes in the network.
[0040] S2. When a transaction occurs on the chain and a block needs to be produced, the node responsible for packaging the block first divides the complete block data into multiple data slices, calculates the hash value of the overall block data and the hash value of each data slice, and then generates simplified block data; the simplified block data includes the block height, block hash value, previous block hash value, block timestamp, and data slice hash list.
[0041] Furthermore, it includes: the node responsible for packing the block first divides the complete block data into multiple data slices in the format of a byte array according to a preset slice size rule.
[0042] S3. After the block data is packaged and generated, the node responsible for packaging the block will broadcast the simplified block data to other nodes in the entire network, and each node will save the simplified block data;
[0043] S4. After the streamlined block data is successfully broadcast, the node responsible for packaging the block selects a number of nodes equal to the total number of nodes in the entire network / 2+1 for each data slice according to the preset node selection rule, and sends each data slice to the distributed storage module of the selected node for storage;
[0044] In some embodiments of the present invention, after the streamlined block data is successfully broadcast, the node responsible for packaging and producing the block is the block data slice generated in step S2. According to a certain rule, a number of nodes (the number of nodes in the entire network / 2 + 1) are selected. The block data slices are then sent to the distributed storage module of the selected nodes for storage. The node selection rule is determined by the consensus algorithm of the blockchain, such as proof of work for PoW, stake ratio for PoS, or random selection for consortium blockchains such as Kafka and Raft.
[0045] S5. After the data slices are successfully distributed, the node responsible for packaging and producing blocks will broadcast the distribution results to the entire network. For each data slice, the node that owns the slice does not perform any additional operations. The node that does not own the slice will store the hash value of the slice and the ID of the nearest node that owns the slice in the distributed hash table of the current node.
[0046] To address the challenges of existing technologies, this invention adds distributed storage and distributed hash table technology to existing blockchain storage. Distributed storage is used to store slices of the original block data that each node needs to store, while the distributed hash table stores the ID and address of each node in the blockchain, as well as the node where the data resides in the distributed storage. Block data is stored in distributed storage as slices, making it impossible to access the original block data through a single slice, effectively protecting data privacy. Each block data is backed up according to the number of nodes in the entire network / 2 + 1. This reduces the total storage capacity by nearly 50%, while ensuring that at least half of the nodes are honest.
[0047] This invention replaces the existing single local storage model of blockchains with a distributed hash table and distributed storage. This significantly improves the overall storage efficiency of all blockchain nodes without compromising the security, reliability, or performance of the blockchain. It also significantly reduces node storage space. This invention ensures the number of backups of all block data, significantly enhancing the reliability and security of the blockchain.
[0048] Figure 3 illustrates the blockchain node storage described in the present invention. Blockchain node storage includes streamlined block data storage, distributed hash table (DHT) fragments, and distributed storage nodes. The streamlined block data storage stores streamlined summary information for all blocks, including block height (block_height), block hash value (block_hash), pre-block hash value (pre_block_hash), block timestamp (block_timestamp), and data slice hash list (slice_hash_list). The DHT fragment stores the ID and address of each node in the blockchain, as well as the node where the data resides in the distributed storage. The S / Kademila DHT distributed hash table protocol can be used to prevent underlying Sybil attacks. Distributed storage stores the original block data slices that each node needs to store. The key for each data entry is the SHA256 or SM3 algorithm hash value of that data.
[0049] As shown in FIG2 , based on the first aspect, in some embodiments of the present invention, the blockchain storage efficiency optimization method further includes the following steps:
[0050] S6. When a node needs to read block data, it searches for the corresponding block data based on the hash value of the block.
[0051] Furthermore, it includes: querying the simplified data of the block in the node storage and obtaining the corresponding data slice hash list; for each data slice hash, first search in the local distributed storage, if found, directly return the corresponding data; if not found, search the distributed hash table for the node ID that owns the slice, then search the distributed hash table for the connection address of the node, and finally initiate a data query to the node to obtain the original data of the slice; splicing the original data of all the slices obtained into complete block data.
[0052] When a node needs to read block data, it searches for it based on the block's hash value. First, it searches the node's storage for the block's reduced data, obtaining a hash list of the data slices it contains. Then, for each data slice hash, it searches the local distributed storage. If the query finds the slice, it returns it directly. If no result is found, it searches the distributed hash table for the node ID that owns the slice. It then searches the distributed hash table for the node's connection address, and finally initiates a data query to the node to retrieve the slice's raw data. Once all slices are retrieved, they are concatenated into the complete block data.
[0053] As shown in Figure 4, in a second aspect, an embodiment of the present application provides an electronic device, which includes a memory 101 for storing one or more programs and a processor 102. When the one or more programs are executed by the processor 102, any method as described in the first aspect above is implemented.
[0054] The system also includes a communication interface 103. The memory 101, processor 102, and communication interface 103 are electrically connected to each other directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes the software programs and modules stored in the memory 101 to perform various functional applications and data processing. The communication interface 103 can be used to communicate signaling or data with other node devices.
[0055] The memory 101 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.
[0056] Processor 102 may be an integrated circuit chip with signal processing capabilities. Processor 102 may be a general-purpose processor, including a central processing unit (CPU) or a network processor (NP). It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0057] In the embodiments provided in this application, it should be understood that the disclosed methods, systems, and methods can also be implemented in other ways. The method and system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the methods, systems, methods, and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a portion of code, and the module, program segment, or a portion of code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.
[0058] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0059] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by the processor 102, implements the method as described in any one of the first aspects above. If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0060] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.
[0061] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present application is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.
Claims
1. A method for optimizing blockchain storage efficiency, characterized in that: The following steps are involved: When a new blockchain node joins the blockchain, it generates a P2P communication certificate for the node. After the proposal to join is passed by consensus, the node certificate hash value is used as the ID, and the node ID and connection address are broadcast and saved in the distributed hash table of all network nodes. When a transaction occurs on the chain and a block needs to be generated, the node responsible for packaging the block first divides the complete block data into multiple data slices, calculates the hash value of the overall block data and the hash value of each data slice, and then generates streamlined block data; After the block data is packaged and generated, the node responsible for packaging the block will broadcast the simplified block data to other nodes in the entire network, and each node will save the simplified block data; After the streamlined block data is successfully broadcasted, the node responsible for packaging the block selects nodes with the number of nodes in the whole network / 2+1 for each data slice according to the preset node selection rule, and sends each data slice to the distributed storage module of the selected node for storage; After the data slices are distributed successfully, the node responsible for packaging and producing blocks will broadcast the distribution results to the entire network.
2. A blockchain storage efficiency optimization method according to claim 1, characterized in that: The following steps are also included: When a node needs to read block data, it searches for the corresponding block data based on the hash value of the block.
3. A blockchain storage efficiency optimization method according to claim 2, characterized in that: The method for searching corresponding block data according to the hash value of the block comprises the following steps: Query the simplified data of the block in the node storage and obtain the corresponding hash list of the included data slices; For each data slice hash, first search in the local distributed storage. If found, the corresponding data is directly returned. If not found, search the distributed hash table for the node ID that owns the slice, then search the distributed hash table for the connection address of the node, and finally initiate a data query to the node to obtain the original data of the slice. The original data of all slices are spliced into complete block data.
4. A blockchain storage efficiency optimization method according to claim 1, characterized in that: The method in which the node responsible for packaging out blocks first divides the complete block data into several data slices includes the following steps: The node responsible for packing blocks first divides the complete block data into multiple data slices in the format of byte array according to the preset slice size rules.
5. A blockchain storage efficiency optimization method according to claim 1, characterized in that: The simplified block data includes block height, block hash value, preceding block hash value, block timestamp and data slice hash list.
6. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 5 is implemented.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Blockchain system capable of simultaneously supporting public blockchain, consortium blockchain and private blockchain and application method
CN107733855A
Blockchain-based cluster security authentication method, node and cluster
CN108259469A
Intellectual property transaction method, system and device based on block chain and computer readable storage medium
CN111444284A
Data storage method, system and device, equipment and storage medium
CN112800450A
Block chain storage efficiency optimization method
CN116701418A
