Block chain data storage device and space optimization method

Abstract

The invention provides a block chain data storage device and a space optimization method, and relates to the field of data storage, and the block chain data storage device comprises a cache module which is used for storing block data indexes of high-frequency access; the distributed storage module is used for dividing the block chain data into dynamic fragments and distributing the dynamic fragments to different nodes for storage; the encryption module is used for encrypting the unstructured data and storing the unstructured data in a local database; the load balancing module is used for dynamically monitoring node loads and optimizing storage resource allocation; and the consensus mechanism module is used for defining data classification, fragmentation rules and migration logic through an intelligent contract, ensuring data consistency by adopting a right proof or PBFT algorithm, and automatically executing a storage rule and data verification. The overall storage cost is reduced, and the storage space is optimized.

Description

Technical Field

[0001] The present invention relates to the field of data storage, and in particular to a blockchain data storage device and a space optimization method. Background Art

[0002] Blockchain technology achieves decentralized storage and verification of data through distributed ledgers, cryptography, and consensus mechanisms. Its core characteristics include immutability, transparency, and decentralization. However, with the expansion of blockchain application scenarios (such as finance, supply chain management, and the Internet of Things), its data storage demand is growing exponentially.

[0003] As blockchain transaction volume grows, nodes need to store complete ledger data, resulting in a sharp increase in local storage pressure. For example, a single node in the Bitcoin network needs to store more than 400GB of data, while the Ethereum network is close to 1TB. This burden is particularly significant for resource-constrained edge nodes or light nodes. The chain storage structure of traditional blockchains requires each node to store all block data, resulting in high storage redundancy and limiting the scalability of the system.

[0004] Therefore, it is necessary to provide a new blockchain data storage device and space optimization method to solve the above technical problems. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a blockchain data storage device and a space optimization method.

[0006] In one aspect, the present invention provides a blockchain data storage device comprising an index cache module for storing frequently accessed block data indexes;

[0007] Distributed storage module, used to divide blockchain data into dynamic shards and distribute them to different nodes for storage;

[0008] Encryption module, used to encrypt unstructured data and store it in a local database;

[0009] Load balancing module, used to dynamically monitor node load and optimize storage resource allocation;

[0010] The consensus mechanism module is used to define data classification, sharding rules and migration logic through smart contracts, adopt proof of stake or PBFT algorithm to ensure data consistency, and automatically execute storage rules and data verification.

[0011] Furthermore, the distributed storage module also includes compatible cross-chain storage, using a universal data protocol to store cross-chain data, and the hash value of the cross-chain data is uploaded to the chain through a multi-chain bridging protocol to ensure data consistency, and a standardized API is set for different chains to call to achieve data sharing and verification.

[0012] Furthermore, the distributed storage module also includes compatible cross-chain storage, using a universal data protocol to store cross-chain data, and the hash value of the cross-chain data is uploaded to the chain through a multi-chain bridging protocol to ensure data consistency, and a standardized API is set for different chains to call to achieve data sharing and verification.

[0013] Furthermore, the structured data in the encryption module is encrypted using AES-256, and the unstructured data is encrypted using RSA-2048, while the hash value is retained on the chain to verify integrity.

[0014] Furthermore, the storage and retrieval of the unstructured data includes the following: file data is encrypted via AES-256 and stored in a distributed file system, only the hash value and metadata of the encrypted file are stored in the blockchain, an Elasticsearch index is established for the unstructured data, and keyword queries are enabled.

[0015] Furthermore, the consensus mechanism module defines data classification rules through smart contracts, which specifically include the following:

[0016] High-frequency data access rules: when the field data volume exceeds the threshold, multiple copies of the data are used for storage;

[0017] For low-frequency data access rules, when the amount of field data is less than the threshold, erasure coding is used for storage;

[0018] Migration logic, when the node load exceeds the threshold, triggers data migration and updates the global reference mapping table.

[0019] Another aspect of the present invention provides a method for optimizing the space of a blockchain data storage device, comprising the following steps:

[0020] Step 1: Use multiple copies of frequently accessed data, prioritize allocating them to low-latency, high-performance nodes, and set the number of copies.

[0021] Step 2: Use erasure coding to split and store infrequently accessed data, store unstructured data in a local database, and only keep hash values ​​and metadata on-chain;

[0022] Step 3: Define storage nodes as virtual objects and generate the optimal dynamic sharding solution by simulating fluid motion and gradient convergence;

[0023] Step 4: When the node load exceeds the threshold, data migration is triggered. The density parameters of adjacent nodes are adjusted to guide new data to the low-load nodes. The migration of low-frequency data from high-load nodes is initiated. The path selection comprehensively considers network latency, transmission volume, and node status to calculate the migration cost.

[0024] Step 5: Build a B+ tree or hash table to accelerate block data retrieval, introduce a full-text search engine for unstructured data, keep the index of frequently accessed data in memory, and store the index of less frequently accessed data on disk.

[0025] In a further method, the dynamic sharding is generated based on an Archimedean optimization algorithm, which optimizes node loads by simulating fluid motion and gradient convergence.

[0026] A further method, wherein the Archimedean optimization algorithm is specifically implemented comprising the following steps:

[0027] S1, initialization phase, randomly generate sharding scheme, calculate node force: Force i =ρ i ·V i -F i , where ρ i is the density of node i, V i is the available storage space, F i For network transmission efficiency;

[0028] S2, the global exploration phase, simulates the random motion of objects in the fluid and traverses different shard combinations to avoid local optimality;

[0029] In the local development phase, shard allocation is adjusted based on gradient convergence, exploration and development are dynamically balanced, and the transfer factor TF is adaptively adjusted to a range of 0.5≤TF≤1.5.

[0030] Compared with related technologies, the blockchain data storage device and space optimization method provided by the present invention have the following beneficial effects:

[0031] 1. This invention significantly improves read and write efficiency through data sharding, compression, and index optimization. The load balancing strategy reduces node resource contention and reduces storage latency.

[0032] 2. The present invention adopts multi-copy storage for high-frequency access data and erasure code segmentation storage for low-frequency access data. Erasure code and multi-copy storage ensure data availability and tolerate partial node failure. Dynamic sharding and distributed storage support linear growth in the number of nodes and data volume. Low-cost storage of low-frequency access data (erasure code, cloud storage) reduces overall storage costs and optimizes storage space.

[0033] 3. This invention uses Archimedean optimization algorithm dynamic sharding, erasure code cold storage, and load balancing migration technology to adopt multiple copies of storage for high-frequency access data and erasure code segmented storage for low-frequency access data, significantly improving the storage efficiency and economy of blockchain, and is suitable for large-scale decentralized scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1This is a system block diagram of the blockchain data storage device provided by the present invention;

[0035] Figure 2 A flowchart of the space optimization method for the blockchain data storage device provided by the present invention;

[0036] Figure 3 This is a flowchart of the Archimedes optimization algorithm provided by the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] Please refer to Figure 1 、 Figure 2 as well as Figure 3 ,in, Figure 1 This is a system block diagram of the blockchain data storage device provided by the present invention; Figure 2 A flowchart of the space optimization method for the blockchain data storage device provided by the present invention; Figure 3 This is a flowchart of the Archimedes optimization algorithm provided by the present invention.

[0039] Example 1

[0040] In the specific implementation process, refer to Figure 1 As shown, the blockchain data storage device includes an index cache module for storing frequently accessed block data indexes;

[0041] Distributed storage module, used to divide blockchain data into dynamic shards and distribute them to different nodes for storage;

[0042] Encryption module, used to encrypt unstructured data and store it in a local database;

[0043] Load balancing module, used to dynamically monitor node load and optimize storage resource allocation;

[0044] The consensus mechanism module is used to define data classification, sharding rules and migration logic through smart contracts, adopt proof of stake or PBFT algorithm to ensure data consistency, and automatically execute storage rules and data verification.

[0045] Optionally, the distributed storage module also includes compatible cross-chain storage, using a universal data protocol to store cross-chain data. The hash value of the cross-chain data is uploaded to the chain through a multi-chain bridging protocol to ensure data consistency, and a standardized API is set for different chains to call to achieve data sharing and verification.

[0046] Optionally, the distributed storage module also includes compatible cross-chain storage, using a universal data protocol to store cross-chain data. The hash value of the cross-chain data is uploaded to the chain through a multi-chain bridging protocol to ensure data consistency, and a standardized API is set for different chains to call to achieve data sharing and verification.

[0047] It is worth noting that the structured data in the encryption module is encrypted using AES-256, and the unstructured data is encrypted using RSA-2048, while the hash value is retained on the chain to verify integrity.

[0048] Optionally, storage and retrieval of unstructured data include the following: file data is encrypted via AES-256 and stored in a distributed file system, only the hash value and metadata of the encrypted file is stored in the blockchain, and an Elasticsearch index is established for the unstructured data with keyword query.

[0049] Optionally, in the load balancing module:

[0050] Load indicators include storage occupancy (S i ), CPU utilization (C i ) and network bandwidth (B i );

[0051] Load threshold: When S i >90% triggers migration;

[0052] The migration cost is calculated as follows:

[0053] Cost ij =α·D ij +β·T ij +γ·L j

[0054] Among them, D ij is the network delay between node i and node j (ms), T ij is the data transmission volume (GB), L j is the current load of node j.

[0055] For example, if the load of node N1 is 95% and 10GB of low-frequency access data needs to be migrated, the candidate node is N2.

[0056] (delay = 20ms, load = 60%), N3 (delay = 30ms, load = 50%);

[0057] Calculate the migration cost (α=0.5, β=1, γ=0.3):

[0058] Cost(N1→N2)=0.5×20+1×10+0.3×60=10+10+18=38

[0059] Cost(N1→N3)=0.5×30+1×10+0.3×50=15+10+15=40;

[0060] Therefore, N2 is selected as the migration target.

[0061] Furthermore, the consensus mechanism module defines data classification rules through smart contracts, including the following:

[0062] High-frequency data access rules: when the field data volume exceeds the threshold, multiple copies of the data are used for storage;

[0063] For low-frequency data access rules, when the amount of field data is less than the threshold, erasure coding is used for storage;

[0064] Migration logic, when the node load exceeds the threshold, triggers data migration and updates the global reference mapping table.

[0065] Example 2

[0066] In a specific implementation process, refer to Figure 2 as well as Figure 3 As shown, a space optimization method for a blockchain data storage device includes the following steps:

[0067] Step 1: Use multiple copies of frequently accessed data, prioritize allocating them to low-latency, high-performance nodes, and set the number of copies.

[0068] Step 2: Use erasure coding to split and store infrequently accessed data, store unstructured data in a local database, and only keep hash values ​​and metadata on-chain;

[0069] Step 3: Define storage nodes as virtual objects and generate the optimal dynamic sharding solution by simulating fluid motion and gradient convergence;

[0070] Step 4: When the node load exceeds the threshold, data migration is triggered. The density parameters of adjacent nodes are adjusted to guide new data to the low-load nodes. The migration of low-frequency data from high-load nodes is initiated. The path selection comprehensively considers network latency, transmission volume, and node status to calculate the migration cost.

[0071] Step 5: Build a B+ tree or hash table to accelerate block data retrieval, introduce a full-text search engine for unstructured data, keep the index of frequently accessed data in memory, and store the index of less frequently accessed data on disk.

[0072] It should be noted that the replica allocation rule is to give priority to nodes with network latency < 50ms and storage cost < 0.2USD / GB.

[0073] It should be noted that in the erasure code segmentation storage of low-frequency access data, the erasure code parameters k = 4 (data blocks), m = 2 (check blocks), and the redundancy rate = 50%;

[0074] For unstructured data storage, the file is divided into 4 data blocks + 2 check blocks, and only the hash value is retained on the chain;

[0075] For example: the original file size is 100MB → divided into 6 blocks (4 data + 2 check blocks), and 4 blocks are stored (total storage capacity = 66.7MB).

[0076] Optionally, a B+ tree is constructed to store the block hash value and offset for each leaf node, and the subtree range for non-leaf nodes.

[0077] It is worth noting that dynamic sharding is generated based on the Archimedean optimization algorithm, which optimizes node load by simulating fluid motion and gradient convergence;

[0078] It should be noted that the specific implementation of the Archimedean optimization algorithm includes the following steps:

[0079] S1, initialization phase, randomly generates a sharding scheme and calculates the node force:

[0080] Force i =ρ i ·V i -F i , where ρ i is the density of node i, V i is the available storage space, F i For network transmission efficiency;

[0081] S2, the global exploration phase, simulates the random motion of objects in the fluid and traverses different shard combinations to avoid local optimality;

[0082] In the local development phase, shard allocation is adjusted based on gradient convergence, exploration and development are dynamically balanced, and the transfer factor TF is adaptively adjusted to a range of 0.5≤TF≤1.5.

[0083] Example:

[0084] Assume 3 nodes (N1, N2, N3), initial parameters:

[0085] N1: ρ=0.1, V=500GB, F=10MB / S→Force=0.1×500-10=40;

[0086] N2: ρ=0.2, V=300GB, F=5MB / S→Force=0.2×300-5=55;

[0087] N3: ρ=0.15, V=400GB, F=8MB / S→Force=0.15×400-8=52;

[0088] The Archimedean optimization algorithm iteratively adjusts shard allocation and ultimately selects N1 to store frequently accessed data (multiple copies) and N2 / N3 to store less frequently accessed data (erasure codes).

[0089] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiment.

[0090] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, from all points of view, the embodiments should be regarded as illustrative and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes that come within the meaning and range of equivalents of the claims are intended to be included in the present invention, and any reference signs in the claims should not be construed as limiting the claims involved.

[0091] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A blockchain data storage device, characterized in that: include: Index cache module, used to store frequently accessed block data indexes; Distributed storage module, used to divide blockchain data into dynamic shards and distribute them to different nodes for storage; Encryption module, used to encrypt unstructured data and store it in a local database; Load balancing module, used to dynamically monitor node load and optimize storage resource allocation; The consensus mechanism module is used to define data classification, sharding rules and migration logic through smart contracts, adopt proof of stake or PBFT algorithm to ensure data consistency, and automatically execute storage rules and data verification.

2. The blockchain data storage device according to claim 1, characterized in that The distributed storage module also includes compatible cross-chain storage, using a universal data protocol to store cross-chain data. The hash value of the cross-chain data is uploaded to the chain through a multi-chain bridging protocol to ensure data consistency, and a standardized API is set for different chains to call to achieve data sharing and verification.

3. The blockchain data storage device according to claim 2, characterized in that: The distributed storage module also includes compatible cross-chain storage, using a universal data protocol to store cross-chain data. The hash value of the cross-chain data is uploaded to the chain through a multi-chain bridging protocol to ensure data consistency, and a standardized API is set for different chains to call to achieve data sharing and verification.

4. The blockchain data storage device according to claim 3, characterized in that The encryption module uses AES-256 to encrypt structured data and RSA-2048 to encrypt unstructured data, while retaining the hash value on the chain to verify integrity.

5. The blockchain data storage device according to claim 4, characterized in that: The storage and retrieval of the unstructured data includes the following: file data is encrypted by AES-256 and stored in a distributed file system, only the hash value and metadata of the encrypted file are stored in the blockchain, and an Elasticsearch index is established for the unstructured data with keyword query.

6. The blockchain data storage device according to claim 5, characterized in that The consensus mechanism module defines data classification rules through smart contracts, which include the following: High-frequency data access rules: when the field data volume exceeds the threshold, multiple copies of the data are used for storage; For low-frequency data access rules, when the amount of field data is less than the threshold, erasure coding is used for storage; Migration logic, when the node load exceeds the threshold, triggers data migration and updates the global reference mapping table.

7. A space optimization method for the blockchain data storage device according to any one of claims 1 to 6, characterized in that: The method comprises the following steps: Step 1: Use multiple copies of frequently accessed data, prioritize allocating them to low-latency, high-performance nodes, and set the number of copies. Step 2: Use erasure coding to split and store infrequently accessed data, store unstructured data in a local database, and only keep hash values ​​and metadata on-chain; Step 3: Define storage nodes as virtual objects and generate the optimal dynamic sharding solution by simulating fluid motion and gradient convergence; Step 4: When the node load exceeds the threshold, data migration is triggered. The density parameters of adjacent nodes are adjusted to guide new data to the low-load nodes. The migration of low-frequency data from high-load nodes is initiated. The path selection comprehensively considers network latency, transmission volume, and node status to calculate the migration cost. Step 5: Build a B+ tree or hash table to accelerate block data retrieval, introduce a full-text search engine for unstructured data, keep the index of frequently accessed data in memory, and store the index of less frequently accessed data on disk.

8. The space optimization method of the blockchain data storage device according to claim 7 is characterized in that: The dynamic sharding is generated based on the Archimedean optimization algorithm, which optimizes the node load by simulating fluid motion and gradient convergence.

9. The space optimization method of the blockchain data storage device according to claim 8, characterized in that: The specific implementation of the Archimedean optimization algorithm includes the following steps: S1, initialization phase, randomly generate sharding scheme, calculate node force: Force i =ρ i ·V i -F i , where ρ i is the density of node i, V i is the available storage space, F i For network transmission efficiency; S2, the global exploration phase, simulates the random motion of objects in the fluid and traverses different shard combinations to avoid local optimality; In the local development phase, shard allocation is adjusted based on gradient convergence, exploration and development are dynamically balanced, and the transfer factor TF is adaptively adjusted to a range of 0.5≤TF≤1.5.

Images