A dynamic sharding storage method in a blockchain system

Abstract

The present invention provides a dynamic sharding storage method in a blockchain system, belonging to the field of blockchain technology, comprising: forming a blockchain system committee; when the data scale of the blockchain system committee exceeds a preset threshold, constructing a subordinate sharded blockchain using a dynamic partitioning method; the leader node and candidate nodes using a hash algorithm to obtain a block hash value of the blockchain system, and sending the block hash value and its corresponding hash algorithm to the nodes in the subordinate sharded blockchain; the nodes in the subordinate sharded blockchain verify the block hash value and its corresponding hash algorithm, and upon successful verification, download the corresponding data from the blockchain system for storage; wherein the nodes and data are all from physical entities with computing resources. The present invention can fully utilize storage resources and improve the scalability of the blockchain system, ensuring the security and integrity of the blockchain system, and can also process large amounts of data, optimizing system performance.

Description

Technical Field

[0001] The present invention belongs to the field of blockchain technology, and in particular relates to a dynamic sharding storage method in a blockchain system. Background Art

[0002] Blockchain technology is a decentralized, distributed ledger. It utilizes cryptography and consensus algorithms to enable participants to securely exchange and verify data without the need for a centralized authority. The core concept of blockchain is to store data records in a block structure, linking these blocks together to form a continuously growing ledger. Each block contains transaction data and a unique hash identifier to ensure data integrity and security. The blockchain's design makes it extremely difficult to modify past data, thereby ensuring data immutability.

[0003] However, traditional blockchain systems face challenges when processing large amounts of data, such as storage capacity limitations, performance bottlenecks, and data access efficiency. Traditional blockchain systems require all nodes to jointly maintain the state and data of the entire blockchain, which limits scalability in large-scale and high-throughput scenarios.

[0004] In order to overcome the problem of node storage capacity limitation in blockchain systems and improve the scalability of the system, it is urgent to propose a dynamic sharding storage method in blockchain systems. Summary of the Invention

[0005] To address the aforementioned technical issues, the present invention proposes a dynamic sharding storage method for a blockchain system. When the data size and network load of a blockchain system exceed a set threshold, blockchain sharding technology is used to construct slave sharded blockchains. Data is transferred from the primary blockchain system to the slave sharded blockchains for storage, reducing the network load of the primary blockchain system and improving its performance and scalability. This method addresses the aforementioned issues with the existing technologies.

[0006] To achieve the above objectives, the present invention provides a dynamic sharding storage method in a blockchain system, comprising the following steps:

[0007] Establishing a blockchain system committee, which includes several leading nodes and candidate nodes;

[0008] When the data size of the blockchain system committee exceeds a preset threshold, a dynamic partitioning method is used to construct a slave shard blockchain;

[0009] The leader node and candidate nodes use a hash algorithm to obtain a block hash value of the blockchain system, and send the block hash value and its corresponding hash algorithm to the nodes in the slave shard blockchain;

[0010] The nodes in the slave shard blockchain verify the block hash value and its corresponding hash algorithm. After verification, the nodes download the corresponding data from the blockchain system for storage;

[0011] Among them, the above nodes and data all come from physical entities with computing resources.

[0012] Optionally, the process of forming a blockchain system committee includes:

[0013] Nodes participating in the blockchain system collect solutions that meet the requirements by executing the POW consensus mechanism. When the number of solutions reaches a preset threshold, the consensus is stopped. Several nodes that have collected the most solutions are selected from the current nodes to serve as the leading nodes of the blockchain system committee, and the remaining nodes serve as candidate nodes of the blockchain system committee.

[0014] Optionally, the blockchain system committee adopts a first-in-first-out strategy, whereby after the first node to become the leader node leaves the blockchain system committee, the first-ranked candidate node becomes the new leader node, and the new member node is added to the blockchain system committee as a candidate node.

[0015] Optionally, when the data size of the blockchain system committee exceeds a preset threshold, the process of constructing a subordinate shard blockchain using a dynamic partitioning method includes:

[0016] The leader node in the blockchain system committee assigns random numbers to member nodes that have not joined the blockchain system committee based on a random number generator, sorts the member nodes in order of number size, and then selects several member nodes in order from small to large to form a slave shard blockchain.

[0017] Optionally, the process of sending the block hash value and its corresponding hash algorithm to the nodes in the slave shard blockchain includes:

[0018] The leader nodes and candidate nodes in the blockchain system committee execute the PBFT consensus mechanism. When 2 / 3 of the committee nodes reach a consensus on the generated block hash value, each leader node signs the block hash value and the corresponding hash algorithm and sends it to the nodes in the slave shard blockchain.

[0019] Optionally, the process of verifying the block hash value and its corresponding hash algorithm by the nodes in the slave shard blockchain includes:

[0020] The nodes in the slave shard blockchain run the SHA1 algorithm to determine whether the block hash value provided by the leader node is the same as the block hash value in the current blockchain system. If they are the same, a verification pass message is sent to the blockchain system committee; if they are different, a new block hash value is requested from the leader node in the blockchain system committee.

[0021] Optionally, after verification, the process of downloading the corresponding data from the blockchain system and storing it includes:

[0022] The nodes in the subordinate shard blockchain maintain the corresponding block data and transmit the maintenance results to the blockchain system committee members, who delete the corresponding block data in the blockchain system.

[0023] The present invention also provides an electronic device, comprising: a memory and a processor; the memory is used to store a program; the processor is used to execute the program to implement each step of the dynamic sharding storage method in the blockchain system.

[0024] The present invention also provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the various steps of the dynamic sharding storage method in the blockchain system.

[0025] Compared with the prior art, the present invention has the following advantages and technical effects:

[0026] Dynamic Data Scale Adjustment: This invention introduces the concept of blockchain sharding, which aims to flexibly organize slave sharded blockchains and dynamically adjust data scale based on changes in data scale. Through dynamic sharded storage, data in a blockchain network can be distributed across multiple slave sharded blockchains, reducing storage pressure on nodes. This approach enables more efficient utilization of storage resources and optimizes blockchain system performance. It also ensures data security and integrity. This design not only enables blockchain systems to better adapt to growing data volumes but also provides an efficient way to increase system capacity and throughput, thereby improving performance and scalability.

[0027] Efficient Utilization of Storage Resources: This invention fully considers nodes with insufficient network computing power. Even nodes with limited storage capacity can still participate in the operation of the blockchain system. Through the committee mechanism and dynamic sharding, nodes can participate based on their storage capacity and computing power, achieving efficient utilization of storage resources. This flexibility allows nodes to contribute according to their own conditions, improving the efficiency and availability of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0029] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of forming a blockchain system committee according to an embodiment of the present invention;

[0031] Figure 3 A schematic diagram of constructing a slave shard blockchain using a dynamic partitioning method according to an embodiment of the present invention;

[0032] Figure 4 This is a schematic diagram of downloading and storing corresponding data from a blockchain system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] Example 1

[0036] like Figure 1 As shown, this embodiment provides a dynamic sharding storage method in a blockchain system, including the following steps:

[0037] Establishing a blockchain system committee, which includes several leading nodes and candidate nodes;

[0038] When the data size of the blockchain system committee exceeds a preset threshold, a dynamic partitioning method is used to construct a slave shard blockchain;

[0039] The leader node and candidate nodes use a hash algorithm to obtain a block hash value of the blockchain system, and send the block hash value and its corresponding hash algorithm to the nodes in the slave shard blockchain;

[0040] The nodes in the slave shard blockchain verify the block hash value and its corresponding hash algorithm. After verification, the nodes download the corresponding data from the blockchain system for storage;

[0041] Among them, the above nodes and data all come from physical entities with computing resources.

[0042] In this embodiment, nodes and data can come from the following physical entities:

[0043] Sensors and network devices: These are the core of the Decentralized Physical Infrastructure Network (DePIN) and include physical assets such as sensors, routers, solar panels, etc. These devices are typically owned and managed by private vendors and are critical for the actual transmission of data, resources, or information within the network.

[0044] WiFi hotspots in wireless networks: As part of DePIN, WiFi hotspots in wireless networks can be part of the physical infrastructure, built by individuals or companies in a decentralized manner, and provide services to anyone.

[0045] Solar home batteries in the energy network: These are also physical infrastructure in DePIN that can be built by individuals or companies and provide energy support to the network.

[0046] Internet of Vehicles equipment: Electric vehicles and their related data transmission equipment can serve as data origins, involving the entire data transmission process from the vehicle terminal to the end user.

[0047] Smart home devices: including various smart appliances and home automation devices, which can be combined with blockchain technology to achieve decentralized management and control.

[0048] Storage networks, communication networks, and data network equipment: These devices can communicate and network independently, forming large physical infrastructure networks to provide network or data services to users.

[0049] Physical Resource Network (PRN): Comprised of location-based physical infrastructure, provider-supplied hardware resources are localized in nature and typically immobile. These resources include connected devices, mobility solutions, energy systems, and geospatial data collectors.

[0050] Digital Resource Network (DRN): Consists of a network of digital resource providers that offer fungible resources that are not constrained by location. These resources include computing power, shared bandwidth, and storage facilities.

[0051] These physical entities are interconnected through blockchain technology, enabling decentralized storage, processing, and transmission of data, while leveraging the immutability and transparency of blockchain to enhance data security and credibility.

[0052] As a specific embodiment, the process of forming a blockchain system committee includes:

[0053] 1) Nodes participate in the blockchain system and collect qualified random numbers (nonce) by executing the POW consensus mechanism.

[0054] 2) When the number of collected nonce reaches a certain threshold, the consensus process stops.

[0055] 3) Select the top m nodes with the most collected solutions from n nodes to serve as the leader nodes of the committee, and the remaining nm nodes are candidate nodes.

[0056] 4) The committee adopts a first-in-first-out strategy. After the first node to become the leader node leaves the committee, the first candidate node in the ranking becomes the new leader node, and the new member nodes are added to the committee as candidate nodes.

[0057] As a specific embodiment, the process of monitoring the data scale of the blockchain system and constructing a slave shard blockchain using a dynamic partitioning method when the data scale exceeds a threshold includes:

[0058] 1) When the average node load in the blockchain system reaches 80%, it is determined that the data scale has exceeded the threshold.

[0059] 2) Randomly select a certain number of nodes that are not members of the committee from the leadership nodes in the committee to form a slave shard blockchain.

[0060] As a specific embodiment, the leader node in the blockchain system committee uses the data segmentation transmission method to segment the block, calculates the hash value of the block using a specific hash algorithm, and signs the block hash value and its corresponding hash algorithm and sends it to the nodes in the slave shard blockchain. The process includes:

[0061] 1) The leader node and candidate nodes in the committee package the hash values ​​from the genesis block to the last block that does not exceed the threshold, and calculate a new hash value through the hash algorithm.

[0062] 2) The leader node and candidate nodes in the committee execute the PBFT consensus mechanism. When 2 / 3 of the committee nodes reach a consensus on the new hash value, the hash value is considered correct.

[0063] 3) Each leader node signs the hash value and the corresponding hash algorithm and sends it to the nodes in the slave shard blockchain for verification.

[0064] As a specific embodiment, the nodes in the slave shard blockchain obtain the data hash value and the signature information of its corresponding hash algorithm and perform verification.

[0065] As a specific embodiment, after verification is passed, the process of downloading the corresponding data from the blockchain system and storing it includes:

[0066] 1) The slave shard blockchain node receives the hash value with the leader node’s signature and its corresponding hash algorithm from the blockchain system and verifies it.

[0067] 2) After verification, the corresponding hash algorithm is run to calculate whether the block hash value provided by the leader node is the same as the block hash value in the current system. If they are the same, a verification pass message is sent to the committee; if they are different, a new block hash value is requested from the leader node in the committee.

[0068] 3) After completing the verification task, the nodes in the slave shard blockchain will continue to maintain the relevant block data and transmit the results to the committee members, who will delete the corresponding block data.

[0069] Example 2

[0070] This embodiment provides a dynamic sharding storage method in a blockchain system, including the following steps:

[0071] Step 1: Form a blockchain system committee.

[0072] Step 2: Monitor the data scale in the blockchain system. When the data scale exceeds the threshold, use the dynamic partitioning method to build a slave shard blockchain.

[0073] Step 3: The leader node in the blockchain system committee uses the data segmentation transmission method to split the block, calculates the hash value of the block through a specific hash algorithm, signs the block hash value and its corresponding hash algorithm, and sends it to the nodes in the slave shard blockchain.

[0074] Step 4: The nodes in the slave shard blockchain obtain the data hash value and its corresponding hash algorithm signature information and verify it.

[0075] Step 5: After verification, download the corresponding data from the blockchain system and store it.

[0076] As a specific example:

[0077] Step 1: If Figure 2 As shown, the blockchain system committee is formed:

[0078] 1) Nodes participating in the blockchain system solve complex mathematical problems by running the POW consensus mechanism.

[0079] 2) When 8 nodes have collected a sufficient number of solutions, the consensus process stops.

[0080] 3) The top three nodes that have collected the most solutions are selected as the leadership nodes of the committee, and the remaining five nodes are candidate nodes.

[0081] 4) The committee adopts a first-in-first-out strategy. The first node to become the leader node leaves the committee, and the first-ranked candidate node becomes the new leader node. New member nodes are added as candidate nodes to join the committee.

[0082] Step 2: If Figure 3 As shown in the figure, the data scale in the blockchain system is monitored. When the data scale exceeds the threshold, a dynamic partitioning method is used to construct a slave shard blockchain.

[0083] 1) Assuming that the average memory of nodes in the blockchain system is 20GB, when the monitored data size exceeds 16GB, it is determined to have exceeded the threshold.

[0084] 2) The leader node in the committee uses a random number generator to assign random numbers to member nodes, sorts the member nodes in order of size, and then selects i nodes in ascending order to form a slave shard blockchain.

[0085] Step 3: The leader node in the blockchain system committee uses the data segmentation transmission method to split the block, calculates the hash value of the block through a specific hash algorithm, signs the block hash value and its corresponding hash algorithm, and sends it to the nodes in the slave shard blockchain.

[0086] 1) The leader node and candidate nodes in the committee calculate a new hash value using the SHA1 algorithm for the data stored from block 1 to block i.

[0087] 2) The leader node and candidate nodes in the committee execute the PBFT consensus mechanism. When 2 / 3 of the committee nodes reach a consensus on the new hash value, the hash value is considered correct.

[0088] 3) Each leader node signs the hash value and the corresponding hash algorithm and sends it to the nodes in the slave shard blockchain for verification.

[0089] Step 4: The nodes in the slave shard blockchain obtain the data hash value and its corresponding hash algorithm signature information and verify it.

[0090] Step 5: Figure 4 As shown, after verification, the corresponding data is downloaded from the blockchain system and stored.

[0091] 1) The slave shard blockchain node receives the hash value and corresponding hash algorithm signed by the leader node from the blockchain system and verifies it.

[0092] 2) After verification, the SHA1 algorithm is run to calculate whether the block hash value provided by the leader node is the same as the block hash value in the current system. If they are the same, a verification pass message is sent to the committee; if they are different, a new block hash value is requested from the leader node in the committee.

[0093] 3) After completing the verification task, the nodes in the slave shard blockchain will continue to maintain the relevant block data and transmit the results to the committee members. The committee members will delete the corresponding block data and continue to expand the blockchain with the i+1th block in the original blockchain system as the current first block.

[0094] Through the above steps, this embodiment implements a method for dynamic sharding storage in a blockchain system, and performs operations such as committee formation, data monitoring, partition construction, and data segmentation and storage according to the illustrated method. This method is operational and flexible, effectively managing the data scale and network load of the blockchain system, and improving system performance and scalability.

[0095] Example 3

[0096] This embodiment also provides an electronic device, including: a memory and a processor; the memory is used to store a program; the processor is used to execute the program to implement each step of the dynamic sharding storage method in the blockchain system.

[0097] Example 4

[0098] This embodiment also provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the various steps of the dynamic sharding storage method in the blockchain system.

[0099] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A dynamic sharding storage method in a blockchain system, characterized in that: The following steps are involved: Establishing a blockchain system committee, which includes several leading nodes and candidate nodes; When the data size of the blockchain system committee exceeds a preset threshold, a dynamic partitioning method is used to construct a slave shard blockchain; The leader node and candidate nodes use a hash algorithm to obtain a block hash value of the blockchain system, and send the block hash value and its corresponding hash algorithm to the nodes in the slave shard blockchain; The nodes in the slave shard blockchain verify the block hash value and its corresponding hash algorithm. After verification, the nodes download the corresponding data from the blockchain system for storage; Among them, the above nodes and data all come from physical entities with computing resources; The process of sending the block hash value and its corresponding hash algorithm to the nodes in the slave shard blockchain includes: The leader nodes and candidate nodes in the blockchain system committee execute the PBFT consensus mechanism. When 2 / 3 of the committee nodes reach a consensus on the generated block hash value, each leader node signs the block hash value and the corresponding hash algorithm and sends it to the nodes in the slave shard blockchain; The process of verifying the block hash value and its corresponding hash algorithm by the nodes in the slave shard blockchain includes: The nodes in the slave shard blockchain run the SHA1 algorithm to determine whether the block hash value provided by the leader node is the same as the block hash value in the current blockchain system. If they are the same, they reply a verification pass message to the blockchain system committee; if they are different, they re-apply for the block hash value from the leader node in the blockchain system committee; After verification, the process of downloading the corresponding data from the blockchain system and storing it includes: The nodes in the subordinate shard blockchain maintain the corresponding block data and transmit the maintenance results to the blockchain system committee members, who delete the corresponding block data in the blockchain system.

2. The method according to claim 1, characterized in that The process of forming a blockchain system committee includes: Nodes participating in the blockchain system collect solutions that meet the requirements by executing the POW consensus mechanism. When the number of solutions reaches a preset threshold, the consensus is stopped. Several nodes that have collected the most solutions are selected from the current nodes to serve as the leading nodes of the blockchain system committee, and the remaining nodes serve as candidate nodes of the blockchain system committee.

3. The method according to claim 2, characterized in that The blockchain system committee adopts a first-in-first-out strategy. After the first node to become the leader node leaves the blockchain system committee, the first candidate node becomes the new leader node, and the new member node is added to the blockchain system committee as a candidate node.

4. The method according to claim 1, wherein When the data size of the blockchain system committee exceeds a preset threshold, the process of constructing a slave shard blockchain using a dynamic partitioning method includes: The leader node in the blockchain system committee assigns random numbers to member nodes that have not joined the blockchain system committee based on a random number generator, sorts the member nodes in order of number size, and then selects several member nodes in order from small to large to form a slave shard blockchain.

5. An electronic device, characterized in that: include: memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the dynamic sharding storage method in the blockchain system according to any one of claims 1 to 4.

6. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the dynamic sharding storage method in the blockchain system according to any one of claims 1 to 4 is implemented.

Images