Block chain storage optimization method based on transaction aggregation backward consensus

Abstract

The invention discloses a block chain storage optimization method based on transaction aggregation backward consensus, and the method comprises the following steps: S1, forming a digraph for historical transfer transactions in a block chain, delimiting a clearing interval, enabling each node in the block chain to participate in the loop search in the digraph, and taking a node reputation value as excitation, the nodes compete for each other, so that more transaction loops can be found in a shorter time; s2, for a competition result of loop search, a deterministic algorithm is adopted to calculate a loop clearing sequence, and transaction multilateral clearing is carried out on all loops according to the sequence; and S3, performing multiple rounds of clearing in the same interval by using an iterative ring buckling algorithm and a continuous transfer transaction simplification mechanism, and taking a clearing result as an aggregation transaction to replace a transaction before clearing so as to quit a block chain storage space. According to the method, the storage space of the block chain is cancelled under the condition that the final transaction execution result is consistent, the expandability of the block chain is improved, and the transaction processing capability is improved.

Description

Technical Field

[0001] The present invention relates to a blockchain storage optimization method based on transaction aggregation and backward consensus. Background Art

[0002] Blockchain technology is a distributed database technology that records and manages data in a decentralized manner. The core features of blockchain technology include immutability, traceability, and transparency, and it is widely used in cross-border payments, smart contracts, supply chain management, identity verification, and other fields.

[0003] Based on the characteristics of blockchain, the academic community has summarized three important characteristics that are difficult to achieve at the same time, which are also known as the "impossible triangle" of blockchain. That is, it is difficult for any blockchain system to have high scalability, high security and strong decentralization at the same time.

[0004] As for the "impossible triangle" summarized by the industry, the lack of some factors will lead to imbalances in other dimensions, which will in turn lead to a significant reduction in the availability of the entire blockchain system. For example, the size of the ledger stored in the full node of the current mainstream public blockchain has expanded to a level that is difficult for home computers to bear. For example, the disk space required to run a full node is at least 1.5TB, and high-performance storage devices such as solid-state drives are required. Therefore, it faces very severe challenges in scalability. The backwardness of blockchain storage scalability will aggravate the degree of network centralization. Therefore, fewer and fewer users can run a complete full node, the participation threshold of the blockchain network is gradually increasing, the number of participants is gradually decreasing, and the monopoly of block generation and accounting rights is intensified. When the degree of decentralization is lower than a certain threshold, the security of the network will also be threatened. It is meaningless to simply consider the security of the network, and it turns to trust a small number of monopolists. Therefore, the scalability research of blockchain storage is crucial and will affect many aspects of the "impossible triangle".

[0005] Specifically, data accumulation on the blockchain is initially executed in the form of transactions in the consensus engine of the node, and the state changes caused by the transactions, the blocks formed by the transactions, and the transactions are written to the database after execution to form persistent storage. In order to realize the legitimacy verification of transactions and states, the mainstream blockchain system often adopts a multi-level KV database at the underlying database level, such as levelDB. When the number of transactions continues to increase, the number of database read and write calls required for state construction and maintenance will further increase, and the disk access performance will be degraded, which will eventually slow down the speed of blockchain transaction processing and cause transaction congestion.

[0006] When the number of transactions in the blockchain reaches a certain level, the storage burden will cause the degradation of the read and write performance of the blockchain nodes, affecting the execution and verification of subsequent transactions. How to effectively manage and optimize the storage in the blockchain system has become an important research direction. Storage optimization aims to reduce system operating costs and improve data processing efficiency, including but not limited to blockchain state data pruning technology, off-chain collaborative storage technology and other technical strategies.

[0007] Blockchain state data pruning technology aims to reduce the overall storage requirements of the blockchain by removing unnecessary state data. In the blockchain, due to the need to consider the state rollback in the event of a smart contract execution failure, the full node needs to save the world state when each block is generated, which leads to rapid data expansion. State data pruning technology attempts to optimize storage space by removing those long-standing historical states and only retaining the state information of the most recent period.

[0008] However, in the process of pruning, data that is critical to understanding the history and state transitions of the blockchain may be accidentally removed. Once this information is lost, it cannot be recovered, which affects the transparency and auditability of the blockchain. In addition, when a new node joins the network, due to the lack of complete historical data, it may need to rely on other nodes to provide additional information to complete synchronization, or build the latest state by replaying all transactions, which not only increases the synchronization time, but also increases the risk of errors during the synchronization process. At the same time, data pruning may potentially increase the risk of data tampering and the possibility of historical state forgery.

[0009] Off-chain collaborative storage technology involves storing part of the blockchain data in an off-chain system to reduce the burden on the main chain. In this way, only critical, transaction-verifying information (such as transaction metadata) is retained on the chain, while larger data blocks are stored off-chain. This model is common in application scenarios that process large data or require frequent updates.

[0010] However, this approach may face data availability, storage reliability, and data consistency issues. Data stored off-chain may face accessibility and availability issues, especially when external storage services encounter downtime or network problems, the data may not be available to network participants in a timely manner. In addition, it relies on the stability and security of external storage services. If the storage service is attacked or damaged, it may affect the integrity and security of the stored data. In addition, the update and synchronization of off-chain data may be inconsistent with the on-chain status, resulting in data inconsistency issues, affecting the overall operating efficiency and security of the system. Summary of the invention

[0011] The purpose of the present invention is to provide a blockchain storage optimization method based on transaction aggregation and backward consensus, which vacates blockchain storage space while ensuring the consistency of the final results of transaction execution, thereby improving the scalability of blockchain and the transaction processing capability.

[0012] To this end, the present invention provides a blockchain storage optimization method based on transaction aggregation and backward consensus, comprising the following steps: S1. For historical transfer transactions that have been placed in the blockchain, they are constructed into a directed graph, and a clearing interval is defined, so that each node in the blockchain participates in the loop search in the directed graph, and the node reputation value is used as an incentive to make each node compete, thereby finding more transaction loops in a shorter time; S2. For the competition results of the loop search, a deterministic algorithm is used to calculate the loop clearing order, and all loops are cleared multilaterally in order; S3. An iterative algorithm and a continuous transfer transaction simplification mechanism are used to carry out multiple rounds of clearing in the same interval, and the clearing results are used as aggregated transactions to replace the transactions before clearing, so as to vacate the blockchain storage space.

[0013] The present invention also provides a blockchain storage optimization device, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned blockchain storage optimization method based on transaction aggregation and backward consensus.

[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned blockchain storage optimization method based on transaction aggregation and backward consensus.

[0015] The present invention optimizes the storage and processing of blockchain through a built-in efficient algorithm. Compared with the existing technology, it not only improves the efficiency of data processing and reduces storage requirements, but also strengthens the security of the system and the consistency of data. These improvements are particularly important for blockchain applications that need to process a large amount of transaction data, and can significantly improve the scalability and maintainability of the system.

[0016] The advantages of this invention over blockchain data clipping technology are:

[0017] 1. Retain key semantic information: Compared with data clipping technology, this invention can effectively retain key semantic information of transactions through intelligent aggregation rather than simple deletion. This ensures the integrity and auditability of the blockchain while optimizing storage space.

[0018] 2. Enhanced data integrity: The present invention not only optimizes storage through loop detection and multilateral clearing, but also increases the organization and optimization of transaction data, thereby maintaining data integrity and consistency and reducing the risk of errors introduced due to clipping.

[0019] 3. Optimized synchronization efficiency: By reducing the total amount of data rather than simply trimming it, the present invention makes the synchronization of new nodes more efficient, because all nodes can obtain a complete and optimized data set without the need for additional data supplementation.

[0020] The advantages of this invention over off-chain collaborative storage are:

[0021] 1. Improve data availability and reliability: This invention does not need to rely on off-chain storage solutions, thus avoiding the problem of off-chain data availability and reliability. All data exists in an optimized form on the chain, ensuring real-time accessibility and high reliability of the data.

[0022] 2. Consistency and security: Since all data processing and storage operations are performed on the chain, the present invention maintains the consistency of data and the inherent security characteristics of blockchain. This eliminates the synchronization delay and consistency problems that may be caused by off-chain storage.

[0023] 3. Reduced Dependency: The present invention reduces the reliance on external storage solutions, avoiding security vulnerabilities and performance bottlenecks that may be caused by third-party services, thereby improving the overall stability and security of the system.

[0024] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 It is a flow chart of the blockchain storage optimization method based on transaction aggregation and backward consensus of the present invention;

[0027] Figure 2 It is the execution flow chart of the loop detection competition consensus;

[0028] Figure 3 It is the execution flow chart of the consensus result verification logic;

[0029] Figure 4 This is an example of a transaction loop, and its clearing order is: 1. Loop ABD-A 2. Loop ABCD-A 3. Loop ACDA;

[0030] Figure 5 It is an example of a multilateral clearing loop breaking process;

[0031] Figure 6 This is an example of a simplified mechanism for equal-amount continuous transfers. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0033] The method according to the present invention is to perform multilateral clearing and aggregation of historical transfer transactions that have been placed in the blockchain, clean up and optimize the transaction process without changing the final execution result of the transaction, vacate the blockchain storage space, and achieve performance expansion of blockchain storage and transaction execution.

[0034] Combined with reference Figures 1 to 6 The blockchain storage optimization method based on transaction aggregation and backward consensus of the present invention mainly includes the following steps S1-S3 when executed.

[0035] S1. Loop detection competition consensus: For the historical transfer transactions that have been placed in the blockchain, they are constructed into a directed graph and intervals are defined. Each node in the blockchain is allowed to participate in the loop search in the directed graph, and the node reputation is used as an incentive to encourage each node to find more transaction loops in a shorter time, so as to carry out multilateral settlement later and save more on-chain storage space.

[0036] S2. Transaction multilateral settlement aggregation. For the results of loop detection competition consensus, a deterministic algorithm is used to calculate the loop settlement order, and multilateral settlement of transactions is performed on all loops in sequence.

[0037] S3. Use the continuous transfer transaction simplification mechanism and iterative algorithm to carry out multiple rounds of clearing in the same interval, and use the clearing results as aggregated transactions to replace the transactions before clearing, thereby saving on-chain storage space.

[0038] Loop Detection Competition Consensus

[0039] Reference Figure 1 For historical transactions that have entered the finalized state, the system selects a certain height interval, maps the hash values ​​of all transfer transactions in the interval to prime numbers, and multiplies these prime values ​​to form an accumulator ACC ALL. Then, the loop detection task is released, that is, a call is initiated to blockchain nodes with certain computing power to perform loop detection on the directed graph composed of all transfer transactions in the selected interval. The one with the earliest submission time and the largest number of detected loops will be rewarded. The reward can be to increase the node's reputation so that subsequent nodes can quickly connect to the node to synchronize data when joining the network.

[0040] Reference Figure 2, the participating competitors will submit the specific information of the loop, including the accumulator value ACC_CYCLE composed of all the edges of the loop, and the hash value of each transaction in the loop. The other blockchain nodes will calculate ACC_CYCLE' based on the hash value of each transaction in the loop through the prime number mapping algorithm, and compare it with the ACC_CYCLE value. If the two are equal, the existence of the values ​​of ACC_CYCLE and ACC_ALL will be verified to verify that the transactions in the loop exist in this interval. Subsequently, the loop will be topologically sorted according to the hash values ​​of each transaction submitted in the loop. If the topological sorting fails, it means that the information submitted by the competitor is indeed a loop, and then the submission time is compared. The one with the earliest submission time will be rewarded, and the loop detected by the competitor will be input into the transaction multilateral settlement algorithm.

[0041] In the present invention, nodes are encouraged to find and reach consensus through competition. This method is different from the traditional loop finding method based on community division or high-complexity algorithm. It encourages participants to develop and use more efficient algorithms to find loops in directed graphs, which may lead to better solutions.

[0042] Multilateral clearing transaction aggregation

[0043] The specific steps of multilateral clearing transaction aggregation are as follows:

[0044] 1. According to the clearing sequencing method, determine the clearing order of all loops found in the previous step;

[0045] 2. Clear each loop in order and break the loop;

[0046] 3. After the loop is broken, the transaction graph is simplified by equal-amount continuous transfer to complete transaction aggregation;

[0047] 4. Replace pre-liquidation transactions with post-liquidation aggregate transactions.

[0048] Among them, the idea of ​​the liquidation sequencing method is as follows:

[0049] 1. Prioritize clearing of loops containing edges with the smallest weights.

[0050] 2. If there are multiple loops that contain the edge with the minimum weight, calculate the difference between the in-degree and out-degree of each node (out-degree - in-degree). This difference represents the net contribution of the node in the liquidation process. A positive difference means that the node has more out-edges, and a negative difference means that the node has more in-edges.

[0051] 3. Among all the loops containing the edge with the smallest weight, select the loop with the largest positive net contribution node for priority liquidation.

[0052] 4. If there are multiple loops containing the same maximum positive net contribution node, the loop with the largest number of maximum positive net contribution nodes will be selected for priority clearing.

[0053] The reason for using this method to determine the liquidation order is: start from the cycle with the largest positive net contribution node, because the nodes in this cycle have more outgoing edges, so prioritizing the liquidation of this cycle may help reduce the number of edges in other cycles.

[0054] The in-degree of a node in a directed graph refers to the number of edges with the node as the end point of the directed edge, and the out-degree refers to the number of edges with the node as the starting point of the directed edge.

[0055] Loop breaking algorithm:

[0056] 1. Enter each loop in turn according to the order of liquidation

[0057] 2. After entering the loop, first find the edge with the smallest weight in the loop (representing the smallest transfer amount) and the corresponding minimum weight

[0058] 3. Deduct the weight of each edge of the loop. The deduction value is the minimum weight found in the previous step. Delete the edge with a weight of 0 after deduction to break the loop.

[0059] 4. For each loop, execute steps 2-3 in order to break the loop until all loops are broken.

[0060] The above gives an optimization algorithm for determining the liquidation order of multiple loops, that is, a deterministic algorithm for liquidation order. The algorithm calculates the net contribution of each node (the difference between the out-degree and the in-degree) and selects the loop with the largest positive net contribution node as the priority liquidation target to ensure that liquidation is carried out in the order with the least remaining transaction volume, which is crucial to achieving storage optimization of the entire system.

[0061] Simplified mechanism for equal-amount continuous transfers

[0062] For the settlement results obtained in the previous step, the transactions are grouped according to the transfer amount, and transactions with continuous transfer relationships are searched in each group and merged into one transaction (i.e., from the initiator of the first transfer to the recipient of the last transfer).

[0063] After executing this step, a new loop may be generated in the interval, so loop detection tasks can continue to be issued, loops can be opened to find competitive consensus, and then multilateral settlement can be carried out, thus achieving iterative settlement in the same interval.

[0064] After the loop is cleared, the present invention also introduces the above-mentioned retrieval and simplification process of the continuous transfer relationship (such as the transfer mode of A->B->C), and reduces the number of transactions by directly replacing them with A->C transactions, that is, simplifying the continuous transfer relationship transactions. This not only further optimizes the storage space, but also may expose new loops again after clearing, providing the possibility for deeper storage optimization.

[0065] In addition, through this method, transactions within the same interval can be cleared multiple times, which improves the effectiveness of clearing, minimizes the number of transactions ultimately retained in the blockchain, and minimizes the on-chain storage space occupancy.

[0066] The present invention also provides a blockchain transaction bilateral clearing storage optimization device, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned blockchain storage optimization method based on transaction aggregation and backward consensus.

[0067] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned blockchain storage optimization method based on transaction aggregation and backward consensus.

[0068] Compared with the prior art, the present invention has the following technical effects:

[0069] 1. Blockchain storage space optimization: Through effective transaction aggregation, the storage requirements of on-chain data are significantly reduced, making blockchain technology more sustainable and widely used. This is especially important for data-intensive blockchain applications.

[0070] 2. Improve blockchain transaction processing efficiency: The optimized transaction processing process reduces potential congestion risks, such as transaction congestion and delay issues, improves the overall efficiency and response speed of the system, and enhances the transaction processing capabilities of the system.

[0071] 3. Improve blockchain scalability: Through simplified transaction structure and optimized storage management, this solution provides a better foundation for blockchain expansion and supports larger-scale network operation.

[0072] 4. The new methods and algorithms provided by this invention may stimulate further research and development in related fields and promote the advancement of the entire blockchain technology.

[0073] The present invention is applicable to payment systems and financial transaction platforms, especially those scenarios that require high transaction throughput and data integrity. The present invention can also be used for supply chain management: it can be used to track and record transactions in complex supply chains, and optimize data storage and verification processes. In addition, the present invention can also be used for data management services: it is applicable to distributed database systems that require large-scale data storage and processing.

[0074] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A blockchain storage optimization method based on transaction aggregation and backward consensus, characterized in that: The following steps are involved: S1. For the historical transfer transactions that have been placed in the blockchain, they are formed into a directed graph and the clearing interval is defined. Each node in the blockchain is allowed to participate in the loop search in the directed graph, and the node reputation is used as an incentive to make each node compete, thereby finding more transaction loops in a shorter time. S2. For the competition results of the loop search, a deterministic algorithm is used to calculate the loop clearing order, and multilateral clearing of transactions is performed on all loops in order; S3. Use the continuous transfer transaction simplification mechanism and iterative algorithm to carry out multiple rounds of clearing in the same interval, and use the clearing results as aggregated transactions to replace the transactions before clearing to free up blockchain storage space.

2. The blockchain storage optimization method based on transaction aggregation and backward consensus according to claim 1 is characterized in that: Step S1 includes: S11. For historical transfer transactions that have been placed in the blockchain, the blockchain system selects a certain height interval as the clearing interval, maps the hash values ​​of all transfer transactions in the interval to prime numbers, and multiplies these prime values ​​to form the accumulator value ACC_ALL, and then releases the loop detection task; S12. The blockchain nodes participating in the competition construct a directed graph of all transfer transactions in the interval, perform a loop search, and calculate the accumulator value ACC_CYCLE for each edge in the found loop; S13, other nodes of the blockchain calculate the accumulator value ACC_CYCLE' according to the hash value of each transaction in the loop through the prime number mapping algorithm; S14. Compare ACC_CYCLE' with the ACC_CYCLE value. If the two are equal, then determine whether ACC_CYCLE is divisible by ACC_ALL. If so, the existence verification is passed, that is, verify that the transaction in the loop exists within the interval. S15. If the verification is passed, the reward given to the searcher is determined based on the result submission time and each node updates the searcher's reputation value to reach a consensus.

3. The blockchain storage optimization method based on transaction aggregation and backward consensus according to claim 1 is characterized in that: Step S2 includes: S21. Determine the clearing order of all loops found in the previous step according to the clearing sequencing method; S22, clearing each loop in order to break the loop; S23. After the loop is broken, the transaction graph is simplified by equal-amount continuous transfer to complete transaction aggregation; S24. Replace the transactions before clearing with the aggregated transactions after clearing.

4. The blockchain storage optimization method based on transaction aggregation and backward consensus according to claim 1 is characterized in that: The liquidation sequencing method in step S21 includes: S211, prioritize clearing the loop containing the edge with the smallest weight; S212. If there are multiple loops that all contain the edge with the minimum weight, the difference between the in-degree and out-degree of each node is calculated. This difference represents the net contribution of the node in the liquidation process. A positive difference means that the node has more out-edges, and a negative difference means that the node has more in-edges. S213, among all the loops containing the smallest weighted edges, select the loop with the largest positive net contribution node for priority liquidation; S214. If multiple loops contain the same maximum positive net contribution node, the loop with the largest number of maximum positive net contribution nodes is selected for priority clearing.

5. The blockchain storage optimization method based on transaction aggregation and backward consensus according to claim 1 is characterized in that: The loop breaking in step S22 includes: S221, enter each loop in turn according to the order of clearing; S222. After entering the loop, first find the edge with the smallest weight in the loop and the corresponding minimum weight, where the smallest weight means the smallest transfer amount; S223, deduct the weight of each edge of the loop, the deducted value is the minimum weight found in the previous step, and delete the edge with a weight of 0 after deduction to break the loop; S224. For each loop, execute steps S222-S223 in order to break the loop until all loops are broken.

6. The blockchain storage optimization method based on transaction aggregation and backward consensus according to claim 1 is characterized in that: The continuous transfer transaction simplification mechanism in step S3 is an equal-amount continuous transfer simplification mechanism, that is, for the settlement result, the transactions are grouped according to the transfer amount, and transactions with continuous transfer relations are searched in each group and merged into one transaction.

7. The blockchain storage optimization method based on transaction aggregation and backward consensus according to claim 1 is characterized in that: The iterative algorithm in step S3 refers to continuing to issue loop detection tasks for new loops that may be generated in the interval after transaction settlement, so as to achieve iterative settlement in the same interval.

8. The blockchain storage optimization method based on transaction aggregation and backward consensus according to claim is characterized in that: Used in payment systems and financial transaction platforms.

9. A blockchain storage optimization device, comprising a computer program, characterized in that: When executed by a processor, the program implements the steps of the blockchain storage optimization method based on transaction aggregation and backward consensus according to any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the blockchain storage optimization method based on transaction aggregation and backward consensus according to any one of claims 1 to 8.

Images