Blockchain data storage consensus method and system for energy data management

Abstract

This invention belongs to the field of blockchain and provides a blockchain data storage consensus method and system for energy data management. The method includes: the ledger node determining its connection status with all other ordinary nodes and updating the distributed hash table; based on the updated distributed hash table, the ledger node synchronizing the content data of all other ordinary nodes according to user needs; and based on the synchronized content data of all other ordinary nodes, the ledger node statistically analyzing the storage changes of each ordinary node and broadcasting the results. This invention implements a synchronization mechanism between the blockchain ledger and off-chain data storage, thereby realizing a consensus method based on blockchain and network-attached storage.

Description

Technical Field

[0001] This invention belongs to the field of blockchain technology, specifically relating to a blockchain data storage consensus method and system for energy data management. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the expansion of the energy data market and the deepening integration of its applications, energy big data centers have accumulated massive amounts of diverse data, bringing challenges to management efficiency and security. Traditional centralized data storage and management systems can no longer meet the demands for distributed, highly available, and highly reliable systems, necessitating a reliable distributed data storage system adapted to energy data storage and management scenarios. Blockchain-based network-attached storage, with its characteristics of independent controllability, distributed nature, and high reliability, perfectly aligns with the needs of energy data storage. To address this issue, a consensus mechanism is introduced based on blockchain and network-attached storage to achieve secure and reliable data management.

[0004] Existing blockchain consensus methods based on anonymous master nodes mainly involve designing a preparation phase, a pre-commit phase, a commit phase, a pre-decision phase, and a decision phase to conduct the consensus process of nodes. This achieves scalable and efficient blockchain data consensus while hiding the identity of the master node. It includes an anonymous node identity authentication mechanism, supports block consensus under anonymous master nodes, and reduces the possibility of master nodes being attacked.

[0005] However, existing technologies still have some problems. The above methods do not take into account the applicability of the method when there are many nodes. When the number of nodes increases significantly, the consensus time will drop sharply, even to the point of being almost unusable. Moreover, most consensus algorithms only consider the ledger layer and do not consider the data storage layer, thus they are not suitable for blockchain-based network-attached storage technology. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a blockchain data storage consensus method and system for energy data management. In a scenario combining blockchain and network-attached storage, this invention utilizes a "Merkle tree + distributed hash table + consistent hashing" approach to achieve data storage routing and location functions. By employing a "location mapping algorithm and ledger nodes acting as management nodes" method, it implements a synchronization mechanism between the blockchain ledger and off-chain data storage, thereby realizing a consensus method based on blockchain and network-attached storage.

[0007] According to some embodiments, the first solution of the present invention provides a blockchain data storage consensus method for energy data management, which adopts the following technical solution:

[0008] Blockchain data storage consensus methods for energy data management include:

[0009] Ledger nodes package transaction ledger data within the blockchain, generate blocks, and broadcast them.

[0010] The accounting node determines its connection status with all other ordinary nodes and updates the distributed hash table.

[0011] Based on the updated distributed hash table, the ledger node synchronizes the content data of all other ordinary nodes according to user needs;

[0012] Based on the content data of all other ordinary nodes after synchronization, the accounting node counts the changes in storage volume of each ordinary node and broadcasts the information.

[0013] Furthermore, the process of determining the accounting node is as follows:

[0014] Initialize all ordinary nodes in the system to obtain the ID of each ordinary node and the corresponding storage capacity of the ordinary node;

[0015] Ordinary nodes with storage exceeding a certain threshold generate random numbers and broadcast them. XOR calculations are performed on the generated random numbers, and location mapping is performed based on the XOR value to determine the accounting node.

[0016] Furthermore, the initialization of all nodes in the system to obtain the ID of each node and the corresponding node storage capacity includes:

[0017] Generate all node IDs in the system by hashing all node IPs;

[0018] Calculate the storage capacity of each node, sort them by node ID from smallest to largest, and obtain the storage capacity information of all nodes.

[0019] Furthermore, nodes with storage exceeding a certain threshold generate and broadcast random numbers, perform XOR calculations on the generated random numbers, and perform location mapping based on the XOR value to determine the accounting node. Specifically:

[0020] Nodes with storage exceeding a certain value generate random numbers and broadcast them.

[0021] After all nodes have finished broadcasting, calculate the XOR value of all generated random numbers;

[0022] Based on the calculated XOR value, the nodes are arranged in ascending order according to the assigned ID value, and each node sequentially maps the number of its storage size to itself.

[0023] Obtain the accounting node.

[0024] Furthermore, the accounting node determines its connection status with all other ordinary nodes and updates the distributed hash table, specifically as follows:

[0025] The accounting node determines its connection status with all other ordinary nodes. If other ordinary nodes do not respond, it is considered a lost node.

[0026] For a lost node, the accounting node updates the distributed hash table and deletes its ID and storage information.

[0027] For newly added nodes, the accounting node generates an ID for them and adds it to the distributed hash table;

[0028] Furthermore, based on the updated distributed hash table, the ledger node synchronizes the content data of all other ordinary nodes according to user needs, specifically as follows:

[0029] The accounting node obtains user needs;

[0030] Based on the acquired user needs, the accounting node determines whether there is a data storage or data retrieval request. If so, the accounting node processes the data storage or data retrieval request.

[0031] If it does not exist, proceed to the next processing step.

[0032] Furthermore, the accounting node processes data storage requests, including:

[0033] Verify whether transaction information corresponding to the stored data exists on the blockchain;

[0034] After successful verification, the hash value of the stored data is calculated, and the consistent rehashing algorithm is used to locate the node corresponding to the stored data for storage. After storage is completed, the node corresponding to the stored data receives a storage incentive.

[0035] Furthermore, the accounting node processes data read requests, including:

[0036] Verify whether the corresponding transaction information exists on the blockchain;

[0037] After successful verification, the hash value of the read data is calculated, and the consistent rehashing algorithm is used to locate the node corresponding to the read data for reading.

[0038] Furthermore, the distributed hash table uses a Merkle file tree, where each leaf node represents a file block, the value of the leaf node is the hash value of the file block, and the value of the non-leaf node is the concatenation of the values ​​of the child nodes before hashing.

[0039] According to some embodiments, the second aspect of the present invention provides a blockchain data storage consensus system for energy data management, employing the following technical solution:

[0040] A blockchain data storage consensus system for energy data management includes users, ordinary nodes, and ledger nodes, wherein the ledger nodes communicate with ordinary nodes and users respectively;

[0041] The accounting node determines its connection status with all other ordinary nodes and updates the distributed hash table.

[0042] Based on the updated distributed hash table, the ledger node synchronizes the content data of all other ordinary nodes according to user needs;

[0043] Based on the content data of all other ordinary nodes after synchronization, the accounting node counts the changes in storage volume of each ordinary node and broadcasts the information.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] This invention uses a location mapping algorithm to solve the data synchronization problem at the ledger layer; it uses an accounting node to replace the management node to solve the data synchronization problem at the storage layer; and it uses an incentive mechanism to help maintain the normal operation of the system.

[0046] Compared with other similar products and solutions, this invention has a wider range of application scenarios and can be applied to blockchain-based network-attached storage systems, which is not available in other solutions. Attached Figure Description

[0047] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0048] Figure 1 This is a flowchart of a blockchain data storage consensus method for energy data management as described in an embodiment of the present invention;

[0049] Figure 2 This is a flowchart of the ledger consensus module at the ledger layer according to an embodiment of the present invention;

[0050] Figure 3 This is a flowchart of the storage consensus module of the storage layer according to an embodiment of the present invention;

[0051] Figure 4 This is a system model diagram of the blockchain data storage consensus method for energy data management described in the embodiments of the present invention. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0053] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0054] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0055] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0056] Example 1

[0057] like Figure 1 As shown, this embodiment provides a blockchain data storage consensus method for energy data management, which includes the following steps:

[0058] Ledger nodes package transaction ledger data within the blockchain, generate blocks, and broadcast them.

[0059] The accounting node determines its connection status with all other ordinary nodes and updates the distributed hash table.

[0060] Based on the updated distributed hash table, the ledger node synchronizes the content data of all other ordinary nodes according to user needs;

[0061] Based on the content data of all other ordinary nodes after synchronization, the accounting node counts the changes in storage volume of each ordinary node and broadcasts the information.

[0062] Specifically, such as Figure 1 As shown, the specific process of the method described in this embodiment includes:

[0063] Step 1: System Initialization

[0064] All nodes in the system are assigned IDs, which are generated by hashing the node's IP address. Simultaneously, the system needs to calculate the storage capacity of each node and sort them by ID in ascending order, completing the preliminary work for the location mapping algorithm.

[0065] Step 2: Generate accounting nodes

[0066] All nodes in the system with storage capacities exceeding a certain value generate random numbers and broadcast them. After all nodes have finished broadcasting, the system calculates the XOR value of all random numbers and applies a location mapping algorithm to them. The node obtained from this calculation is the accounting node.

[0067] Step 3: Ledger-level synchronization

[0068] Ledger nodes package blockchain ledger data, generate blocks, and broadcast them.

[0069] Step 4: Storage layer synchronization

[0070] The ledger node connects all nodes. For a lost node, it updates the distributed hash table, deleting its ID and storage information. For a newly added node, it generates an ID for it and adds it to the distributed hash table. At the same time, the ledger node needs to handle data storage and read requests. For read requests, it locates the specific node and reads the data. For storage requests, it verifies that the corresponding transaction information exists on the blockchain and then locates the corresponding node for storage.

[0071] Step 5: Calculate the storage volume and obtain the corresponding incentive value.

[0072] The ledger node calculates the storage volume of each node and the total storage volume of the system. Based on the contribution of storage in this consensus, the corresponding incentive is calculated using the incentive formula in Module 3.

[0073] After this consensus process concludes, we will proceed to the next step.

[0074] The method described in this embodiment is divided into three modules: (1) ledger consensus module, (2) storage consensus module, and (3) incentive mechanism module. Module 1 and Module 2 are responsible for the data consensus and synchronization mechanism of the ledger layer and storage layer, respectively. Module 3 is in a leading relationship with Module 1 and 2 and is responsible for maintaining the normal operation of these two modules.

[0075] In Module 1, at the beginning of each time step, all nodes with storage exceeding a certain threshold generate random numbers and broadcast them. They then publicly calculate the XOR value of these random numbers, and the location corresponding to the final result becomes the storage location assigned to the ledger node. This node becomes the ledger node and is responsible for packaging the data. Finally, the ledger node organizes the data stored by each node, reorders the storage locations of each node, calculates the total data volume of the system, and then broadcasts the information to proceed to the next time step.

[0076] The threshold is half of the average storage of a node.

[0077] The position mapping algorithm is used in Module 1.

[0078] The position mapping algorithm maps numbers to specific nodes, denoted by g(·). All random numbers are XORed. In the position mapping algorithm, nodes are arranged in ascending order of their assigned ID values, and each node sequentially maps numbers equal to its storage size to itself. For node n1, all numbers from 1 to C1 are mapped to node n1, i.e., g(x) = n1, x ∈ [1, C1], where C1 is the storage size of that node. This process continues until node n... k The storage capacity is C k Then it has already been used. All previous numbers, therefore from All numbers are mapped to node n k The selection of accounting nodes is achieved using a location mapping algorithm.

[0079] In Module 2, the ledger nodes identified in the ledger consensus module are responsible for managing node functions and connecting all nodes. For nodes whose connections are interrupted or timed out, after a certain number of attempts, they are deemed to have left the network, and their storage location and distributed hash table are reassigned. Simultaneously, requests to store and retrieve data are processed. To retrieve data, the distributed hash table is used to locate the data, and a request is sent to the storage node. For stored data, after checking for the existence of transaction information on the blockchain, the data's hash is calculated to locate the data, and then it is sent to the corresponding node.

[0080] Module 2 uses a Merkle file tree, where each leaf node represents a file block. The value of a leaf node is the hash value of the file block, while the value of a non-leaf node is the concatenation of the values ​​of its child nodes before hashing.

[0081] V parent =H(V) lchild ||V rchild )

[0082] Using this Merkle file tree, the dependency relationships of a file block can be quickly verified, facilitating fast file reading and recovery. Furthermore, by constructing a Merkle file tree for large files and using the root hash as the hash value, the time required is significantly reduced compared to directly calculating the hash value. The distributed hash table is used for routing to locate the file storage location; it is maintained by each node and stored in a table format.

[0083] Module 2 uses a consistent hashing technique that can be used to locate multiple file replicas: consistent rehashing.

[0084] Consistent rehashing is a special type of consistent hashing technique that optimizes the hash calculation of a file. It concatenates the file's hash value with its replica number, performs another hash operation, and the result can be used to locate the node containing that replica number.

[0085] V file =H(V) origin ||rep),

[0086] Among them, V file It is the hash value of the file block `file` in the distributed hash table, V origin It is the original hash value of the file block, and rep is the copy number, which is represented in numerical form.

[0087] In this way, a single distributed hash table can be used to locate the node positions of multiple different replicas, saving storage space compared to using multiple hash tables.

[0088] By using Merkle file trees to store file hashes, employing consistent rehashing as a consistent hashing technique, and combining this with a distributed hash table, file storage and retrieval functions can be achieved.

[0089] Module 3 is the incentive mechanism. The incentive mechanism is a key technology in blockchain networks, used to maintain the normal operation of the network. Blockchain operation relies on the voluntary maintenance of nodes; without direct rewards, nodes are inclined to maliciously attack the network. The incentive mechanism in blockchain encourages users to participate in maintaining network operation through rewards, providing benefits for certain specific actions. Incentives are designed to better motivate storage nodes to actively store data.

[0090] This method employs two types of incentives: consensus incentives and access incentives. Consensus incentives reward the nodes that record data, encouraging them to act honestly and reliably; access incentives reward the nodes that store data, encouraging them to actively help store data.

[0091] In Module 3, the access stimulus determination formula is:

[0092]

[0093] in, This represents the reward value that node j receives on transaction i. C represents the amount of data in transaction i for the j-th node. j F represents the storage size of node j. i This represents the cost of transaction i.

[0094] The consensus method based on blockchain and network-attached storage includes three roles: ordinary nodes, ledger nodes, and users. Ledger nodes are generated by ordinary nodes through calculation.

[0095] The following example uses four ordinary nodes to illustrate an embodiment of the present invention.

[0096] The provided embodiments include the following steps:

[0097] (1) Initialization

[0098] Each node in the system confirms with each other, assigns IDs, and records storage information. Preparations are then made for the location mapping algorithm.

[0099] Assuming the IDs of nodes A, B, C, and D are aa072e, ac7714, 15112c, and 237dac respectively, and their storage sizes are 3, 4, 1, and 2 respectively, then for numbers 1-10, number 1 is mapped to node C, numbers 2 and 3 are mapped to node D, numbers 4, 5, and 6 are mapped to node A, and numbers 7, 8, 9, and 10 are mapped to node B.

[0100] (2) Generate and confirm accounting nodes

[0101] Each node generates a random number and broadcasts it. Upon receiving random numbers from other nodes, it performs an XOR operation, and after location mapping, obtains the accounting node. The accounting node then sends an acknowledgment broadcast.

[0102] Suppose nodes A, B, C, and D generate random numbers 5, 4, 7, and 3 respectively. XORing these four numbers yields the number 5. According to the initialization process in the previous step, the position of number 5 is mapped to node A. After confirming itself as the ledger node, node A broadcasts the message.

[0103] (3) The ledger node packages the blockchain data and broadcasts it.

[0104] The ledger node packages the transaction information within this period into blocks and broadcasts them.

[0105] Node A packages the transactions in its own transaction pool, forms a block, and then broadcasts the block.

[0106] (4) The accounting node is responsible for maintaining the stored data and connection status.

[0107] The accounting node connects to other nodes, maintains connection status information, and synchronizes data.

[0108] Node A connects to nodes B, C, and D, and checks the connection status. If node D loses its connection, after attempting to reconnect a certain number of times, node A determines that it has disconnected and exited the network, and clears its storage to zero.

[0109] Meanwhile, node A is responsible for handling data storage and retrieval requests. Suppose user E requests to read file F. Node A first needs to verify from the blockchain ledger that user E has submitted a transaction to store file F. After successful verification, it uses a hash algorithm to calculate the hash value of file F and then searches for it using a consistent rehashing algorithm.

[0110] Assuming the hash value of file F is 11223f, according to the consistent hashing algorithm, its corresponding storage node is node B (11223f%4=3, corresponding to the fourth node in ascending order of ID). Node A then retrieves data from node B and returns it to the user.

[0111] Similarly, assuming user E requests to store file G, node A first needs to verify from the blockchain ledger that user E has submitted a transaction to store file G. After successful verification, it uses a hash algorithm to calculate the hash value of file G and then searches for it using a consistent rehashing algorithm. Assuming the hash value of file F is 3d8a9c, according to the consistent rehashing algorithm, its corresponding storage node is node C (3d8a9c % 4 = 0, corresponding to the first node in ascending order of ID). Node A then transmits the data to node C, and after node C completes the storage, it notifies the user of completion. After the transmission is complete, node C receives an incentive.

[0112] (5) Statistical storage information for accounting nodes

[0113] The accounting node compiles and broadcasts information on changes in storage volume across all nodes.

[0114] Node A needs to collect and broadcast information on the changes in storage capacity of each node. In the previous step, node D disconnected, so its storage capacity information needs to be cleared. Additionally, in the previous step, node C stored file F, so the size of file F needs to be added to node C's storage capacity. After broadcasting the information, this process ends, and preparation begins for the next process.

[0115] Example 2

[0116] This embodiment provides a blockchain data storage consensus system for energy data management, including users, ordinary nodes, and ledger nodes, wherein the ledger nodes communicate with ordinary nodes and users respectively;

[0117] The accounting node determines its connection status with all other ordinary nodes and updates the distributed hash table.

[0118] Based on the updated distributed hash table, the ledger node synchronizes the content data of all other ordinary nodes according to user needs;

[0119] Based on the content data of all other ordinary nodes after synchronization, the accounting node counts the changes in storage volume of each ordinary node and broadcasts the information.

[0120] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0121] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0122] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0123] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A blockchain data storage consensus method for energy data management, characterized in that, The application relates to a distributed storage system based on blockchain, which comprises the following steps: An accounting node packs transaction ledger data in a blockchain, generates a block and broadcasts the block; The accounting node judges the connection state of all other normal nodes and updates a distributed hash table; Based on the updated distributed hash table, the accounting node synchronizes content data of all other normal nodes according to user demand; Based on the synchronized content data of all other normal nodes, the accounting node counts the storage capacity change information of each normal node and broadcasts the information; Based on the updated distributed hash table, the accounting node synchronizes content data of all other normal nodes according to user demand, specifically as follows: the accounting node acquires user demand; based on the acquired user demand, the accounting node judges whether there is a data storage or data reading request, and if there is, the accounting node processes the data storage or data reading request; If not, the next processing flow is entered; The accounting node processes the data storage request, which comprises the following steps: verifying whether there is transaction information corresponding to the storage data on the blockchain; after successful verification, the hash value of the storage data is calculated, and the storage data is located according to a consistent rehashing algorithm to store the data in the corresponding node; after the storage is completed, the node corresponding to the storage data obtains an access incentive; The accounting node processes the data reading request, which comprises the following steps: verifying whether there is transaction information corresponding to the reading data on the blockchain; after successful verification, the hash value of the reading data is calculated, and the reading data is located according to a consistent rehashing algorithm to read the data in the corresponding node.

2. The blockchain data storage consensus method for energy data management of claim 1, wherein, The determination process of the accounting node is specifically as follows: All normal nodes in the system are initialized to obtain the ID of each normal node and the corresponding normal node storage capacity; The normal node with a storage capacity above a certain threshold generates a random number and broadcasts the random number, the random number is calculated through XOR, and the accounting node is determined based on the calculated XOR value.

3. The blockchain data storage consensus method for energy data management of claim 2, wherein, The initialization of all nodes in the system to obtain the ID of each node and the corresponding node storage capacity comprises the following steps: The ID of all nodes in the system is generated through hash operation based on the IP of all nodes; The storage capacity of each node is counted, and the storage capacity information of all nodes is obtained by sorting the node IDs from small to large.

4. The blockchain data storage consensus method for energy data management of claim 2, wherein, The node with a storage capacity above a certain value generates a random number and broadcasts the random number, the random number is calculated through XOR, and the accounting node is determined based on the calculated XOR value. The node with a storage capacity above a certain value generates a random number and broadcasts the random number; After all nodes broadcast, the XOR value of all generated random numbers is calculated; Based on the calculated XOR value, the nodes are arranged in the order of the assigned ID values from small to large, and each node sequentially maps the storage capacity size of itself to itself; The accounting node is obtained.

5. The blockchain data storage consensus method for energy data management of claim 1, wherein, The accounting node judges the connection state of all other normal nodes, and updates a distributed hash table, specifically as follows: The accounting node judges the connection state of all other normal nodes, and if the other normal nodes do not respond, they are lost nodes. The accounting node updates the distributed hash table for the lost node, deleting the ID and storage information thereof; The accounting node generates an ID for the newly added node and adds the node to the distributed hash table.

6. The blockchain data storage consensus method for energy data management of claim 5, wherein, The distributed hash table adopts a Merkle file tree, in which each leaf node represents a file block, the value of the leaf node is the hash value of the file block, and the value of a non-leaf node is the hash value of the concatenation of the values of the child nodes.

7. A blockchain data storage consensus system for energy data management, characterized in that, The system comprises a user, a common node, and an accounting node, wherein the accounting node communicates with the common node and the user respectively; The accounting node judges the connection state with all other common nodes and updates the distributed hash table; Based on the updated distributed hash table, the accounting node synchronizes the content data of all other common nodes according to the user demand; Based on the synchronized content data of all other common nodes, the accounting node broadcasts the storage change information of each common node; Based on the updated distributed hash table, the accounting node synchronizes the content data of all other common nodes according to the user demand, specifically as follows: the accounting node obtains the user demand; based on the obtained user demand, the accounting node judges whether there is a data storage or data reading request, and if there is, the accounting node processes the data storage or data reading request; If not, the next processing flow is entered; The accounting node processes the data storage request, including: verifying whether there is transaction information corresponding to the storage data on the block chain; after successful verification, calculating the hash value of the storage data and finding the node corresponding to the storage data according to the consistent rehash algorithm to store the data; after the storage is completed, the node corresponding to the storage data obtains the access incentive; The accounting node processes the data reading request, including: verifying whether there is transaction information corresponding to the reading data on the block chain; after successful verification, calculating the hash value of the reading data and finding the node corresponding to the reading data according to the consistent rehash algorithm to read the data.

Images