A high-efficiency blockchain query system based on Gskiplist and its dynamic construction method
By introducing Gskiplist and Merkle tree structures into the blockchain system, the grouped skip lists are dynamically adjusted to adapt to user query needs, solving the problem of efficient querying of large-scale data in the blockchain system, and realizing efficient and complex query and decentralized query services.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANKAI UNIV
- Filing Date
- 2023-11-02
- Publication Date
- 2026-05-05
AI Technical Summary
Existing blockchain systems struggle to support efficient query operations in large-scale data scenarios, especially complex queries such as Top-k queries and range queries. Furthermore, external database storage solutions suffer from issues related to immutability and decentralization.
A high-efficiency blockchain query system based on Gskiplist is adopted. By constructing a grouped skip list Gskiplist, its structure is dynamically adjusted to adapt to user query needs. Combined with the Merkle tree and Bloom filter of the blockchain, a high-efficiency complex query service is achieved.
While ensuring the decentralization and immutability of the blockchain, it improves query efficiency, supports complex query needs, and avoids security vulnerabilities of external databases.
Smart Images

Figure CN117472959B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer science and focuses on improving the efficiency of blockchain query and retrieval. Background Technology
[0002] Blockchain is essentially a decentralized and tamper-proof distributed database with broad prospects in finance, the Internet of Things, public services, notarization, digital copyright, and traceability. As the amount of data stored and the number of users on the blockchain gradually increase, the throughput requirements of the blockchain system are also constantly growing, but the following problems exist:
[0003] 1. With the rapid growth of blockchain data volume, the existing blockchain system structure and algorithms are struggling to support efficient data query operations in large-scale data scenarios. Under the existing blockchain structure and algorithms, a query involves accessing multiple underlying disk files, resulting in a large amount of disk I / O and significantly impacting the performance of query operations.
[0004] 2. With the rapid growth of blockchain data, existing blockchain systems struggle to efficiently handle complex queries, such as Top-k queries, range queries, and equality queries. However, there is a significant demand for complex blockchain queries in real-world scenarios. For example, blockchain investors might want to know the total transaction volume, maximum transaction volume, total transaction volume, and average transaction volume for the most recent quarter. They hope to use this data to estimate the current operational status of the blockchain system and determine whether it is suitable to buy or sell. Therefore, users would like blockchain systems to provide interfaces for querying Ethereum data within a specific range. However, currently, blockchain systems can only retrieve range data one by one from the underlying database, such as LevelDB. This method is not only inefficient but also wastes computational resources.
[0005] To address the aforementioned issues, the primary solution currently involves building external databases to store blockchain data. For example, EtherQL synchronizes and parses blockchain data from the blockchain system. This parsing process redesigns the data storage format and stores the results in MongoDB. MongoDB is a high-efficiency and scalable NoSQL database, significantly improving the efficiency of blockchain data queries and supporting various complex queries. While this solves the two problems mentioned above, using external databases has the following limitations: 1) It cannot guarantee the immutability and decentralization of data, and it also increases system overhead; 2) Query efficiency is determined by the external database, potentially leading to single points of failure, data loss, and data tampering, thus posing significant security vulnerabilities to off-chain storage.
[0006] Other typical solutions are as follows:
[0007] Reference 1: Zheng Haohan, Shen Derong, Nie Tiezheng, et al. Queryability optimization of blockchain systems for hybrid indexes [J]. Computer Science, 2020, 47(10):301-308.
[0008] This paper proposes a novel solution: First, blockchain data is divided into different attributes. Then, based on these attributes, and combining the blockchain's Merkle tree and various index structures, a new index—the MHerkle tree—is proposed. Each block is generated by sorting all transactions within it according to time or gas value. The sorted transactions are then constructed into a corresponding Merkle prefix tree and stored in the block body. This ensures that transactions within a block are ordered in a specific order. A Bloom filter is also added for quickly checking if a transaction is present in the block body. This structure enhances the blockchain's query performance while fully guaranteeing its immutability. Finally, an MHerkle tree index construction algorithm is designed, and query algorithms based on different attributes and range queries are proposed based on the index. The advantages of this solution are its ability to perform local range searches and top-k searches, and the use of a Bloom filter improves query efficiency. The disadvantages are that it cannot perform global range searches and top-k searches, and the Bloom filter also has the problem of false positives, resulting in limited efficiency improvements.
[0009] Reference 2: Zhang Hong, Wei Zhongqi. Query model of blockchain system based on two-level index mechanism [J]. Computer Applications, 2022, 42(S2):129-134.
[0010] This paper proposes a query model based on account public keys and smart contracts. The proposed model establishes a secondary index on the hash values of blocks and transactions, enabling users to quickly and accurately retrieve transaction information using account information. The model adds this secondary index as a special node to the blockchain network and establishes an access control mechanism through smart contracts, ensuring its secure maintenance and sharing by all users. The advantages of this scheme are that it provides a novel method for quickly obtaining a user's historical transaction data while maintaining the decentralized nature of the index. The disadvantage is that it simply chains together all of a user's transaction data and cannot satisfy advanced queries on a user's historical transactions, such as tok-k queries and range queries based on time or gas value. Summary of the Invention
[0011] The purpose of this invention is to improve the efficiency of blockchain query and retrieval. This invention presents a high-efficiency blockchain query system based on Gskiplist and a dynamic construction method.
[0012] Gskiplist, as described in this invention, is a grouped skip list, or Gskiplist for short. It's a novel skip list that supports a range of key values, and its structure can be dynamically adjusted based on user query requirements. Each node in the grouped skip list contains not just a single key value, but a range of keys ([KeyLeft, KeyRight)). KeyLeft is the left key, and KeyRight is the right key. Therefore, each node points to a set of MetaD nodes, and each MetaD node points to a transaction ID (TransactionID) and a block ID (BlockID) in the blockchain. Furthermore, the multi-level linked "Node tree" composed of nodes can dynamically upgrade or downgrade a node based on user query requirements.
[0013] This invention employs the Gskiplist technology to establish a Gskiplist instance with a specified set of query fields on the existing blockchain platform, namely the blockchain and database. The Gskiplist instance is initialized with an initial preset and its structure is dynamically adjusted based on the user's query requirements, thereby providing a highly efficient complex query service for the blockchain system.
[0014] The technical solution of the present invention:
[0015] A high-efficiency blockchain query system based on Gskiplist is disclosed, comprising a blockchain subsystem, a "grouped skip list" subsystem, and a database. The blockchain subsystem contains all "transactions" and "blocks." Each "transaction" includes at least a transaction ID (TransactionID), transaction time, transaction participants, transaction header hash, and transaction content information. Each "block" contains transactions within a given time period. Blocks are linked together in a chain based on their creation time. Each "block" includes at least a block ID (BlockID), block number, block start and end times, the previous block ID, the Merkle root of the contained "transaction" set, the block header hash, and information about the contained "transaction" set. All "transaction" and "block" data in the blockchain subsystem are stored in the database.The "Grouped Skip List" subsystem, abbreviated as Gskiplist, contains a multi-level "Node Tree" composed of "Nodes" and multiple "MetaD Node Sets" composed of "MetaD Nodes". Each "Node" contains at least the Node ID (NodeID), the left key (KeyLeft), the right key (KeyRight), the ID of the first MetaD node (MetaDID), the search count (SCount), the pointer array Prev of the previous "Node", and the pointer array Next of the next "Node". Each "Node" contains the corresponding "MetaD Node Set" within the range [KeyLeft, KeyRight) that is greater than or equal to the left key and less than the right key. Each "Node" also constructs multiple levels of chained relationships with other "Nodes" through the pointer arrays Prev and Next. Each "MetaD Node" contains at least the MetaDID, the key, the NodeID of the corresponding "Node", the transaction ID in the corresponding blockchain (TransactionID), and the corresponding block. In the blockchain, the block ID is called BlockID, the MetaDID of the previous MetaD node is called Prev, and the MetaDID of the next MetaD node is called Next. The Gskiplist subsystem also includes a Gskiplist parameter file to define the multi-level Node tree. The Gskiplist parameter file must contain at least the Gskiplist ID (GSL_ID), the Gskiplist search field set (GSL_SearchFields), the maximum level N of the Node tree, and the initial level of the Node tree. The initial preset level Q, the initial preset rule type R of the "Node tree", the "Node" upgrade threshold U, the "Node" downgrade threshold D, the "MetaD" fission threshold S, the "MetaD" merge threshold M, the maximum memory space occupied G, and the persistent storage decision threshold P are all information. All "Node" and "MetaD" data in the Gskiplist subsystem are stored in the database. The database is used to store transaction data and block data of the blockchain subsystem, and also to store "Node" and "MetaD" data of the Gskiplist subsystem.
[0016] This invention also provides a dynamic construction method for a high-efficiency blockchain query system based on Gskiplist, the method comprising the following specific steps:
[0017] Step 1: Start the system and set the "Gskiplist parameter file" to be read;
[0018] Step 2: Read the "Gskiplist parameter file" to obtain the Gskiplist ID (GSL_ID), the Gskiplist search field set (GSL_SearchFields), the maximum level of the "Node tree" (N), the initial preset level of the "Node tree" (Q), the initial preset rule type of the "Node tree" (R), the "Node" upgrade threshold (U), the "Node" downgrade threshold (D), the "MetaD" splitting threshold (S), the "MetaD" merging threshold (M), the maximum memory space occupied (G), and the persistent storage decision threshold (P).
[0019] Step 3: Read the blockchain data and build the initial preset "group skip list" subsystem. The specific steps are as follows:
[0020] Step 3.1: Based on the GSL_SearchFields set of search fields in step 2, read the blockchain data to construct the level 0 "Node tree" and multiple "MetaD node sets";
[0021] Step 3.2: Based on the initial preset level Q of the "Node Tree" in Step 2, construct the initial preset "Node Tree" from level 1 to level Q on the basis of the level 0 Node Tree;
[0022] Step 4: The system enables blockchain query services to be provided to external parties;
[0023] Step 5: Monitor the generation of new blocks in the blockchain subsystem, and simultaneously monitor and receive "query requests from the user".
[0024] Step 6: Determine the type of the listening response. If it is "a new block generated in the blockchain subsystem", proceed to step 17; if it is "a query request from the user", proceed to step 7.
[0025] Step 7: Parse the user's query request;
[0026] Step 8: Convert the query request in step 7 into the key value range [Vsb, Vse) to be queried;
[0027] Step 9: Search step by step from the "highest level" of the Gskiplist subsystem down to level 0, finding all "Nodes" corresponding to the key value range [Vsb, Vse) given in step 8. The "highest level" of the Gskiplist subsystem starts at K, but changes dynamically as the structure of the Gskiplist subsystem changes. The [KeyLeft, KeyRight) of each retrieved "Node" must have a non-empty intersection with the key value range [Vsb, Vse) given in step 8.
[0028] Step 10: Among all the “Nodes” found in step 9, further search for all “MetaD Nodes” corresponding to the key value range [Vsb, Vse) given in step 8. That is, the key of the retrieved “MetaD Node” must be within the key value range [Vsb, Vse) given in step 8.
[0029] Step 11: Based on the transaction ID (TransactionID) and block ID (BlockID) information of all the "MetaD nodes" retrieved in step 10, further search for the relevant "blockchain data" in the blockchain.
[0030] Step 12: For all the "blockchain data" found in step 11, further calculate the "final query result" of the "user's query request";
[0031] Step 13: For all the "Nodes" found in step 9, increment the "retrieval count (SCount)" of each "Node" by 1;
[0032] Step 14: Dynamically adjust the structure of the Gskiplist subsystem using synchronous or asynchronous methods. The specific steps are as follows:
[0033] Step 14.1: Perform "dynamic structural adjustment" on the Gskiplist subsystem using synchronous or asynchronous methods, that is, dynamically adjust the upgrade or downgrade of "Node nodes" and dynamically adjust the splitting or merging of "MetaD node sets";
[0034] Step 14.2: Perform "persistent storage decision and implementation" on the Gskiplist subsystem using synchronous or asynchronous methods, and save all in-memory data of the Gskiplist subsystem to the database;
[0035] Step 15: Return the "final query result" calculated in step 12 to the user.
[0036] Step 16: Determine whether the system should continue running. If the system continues running, proceed to step 5; if the system does not continue running, proceed to step 18.
[0037] Step 17: Read the "new block" of the blockchain and perform "content update" on the Gskiplist subsystem, that is, update multiple "MetaD node sets" and "Node node trees" according to the transaction information in the "new block"; then jump to step 5 to continue execution;
[0038] Step 18: Implement persistent storage for the Gskiplist subsystem using a synchronous method, saving all in-memory data of the Gskiplist subsystem to the database;
[0039] Step 19: System execution complete.
[0040] Furthermore, according to the access method provided by this invention, the construction of the level 0 "Node tree" and multiple "MetaD node sets" for the Gskiplist subsystem in step 3.1 involves sequentially reading blockchain blocks and transaction data, and establishing the level 0 "Node tree" and multiple "MetaD node sets" based on the Gskiplist search field set, i.e., GSL_SearchFields, in step 2. This can be achieved using the following steps:
[0041] Step 3.1.1: Read the block data from block 0 to the "maximum ID block" in the blockchain sequentially. Each block read is abbreviated as Bi, where 0 <= i < "maximum ID block". Perform the following processing on each block Bi;
[0042] Step 3.1.2: Traverse each transaction in block Bi, denoted as Tj, where 0 <= j < "the maximum number of transactions in block Bi". Perform the following processing on each transaction Tj;
[0043] Step 3.1.3: Calculate the key and value (key, value) for transaction Tj based on the set of search fields of Gskiplist in step 2, namely GSL_SearchFields, and record the result as (Kj, Vj).
[0044] Step 3.1.4: Create a MetaD node MDj for transaction Tj. The key value of MetaD node MDj is Kj, the TransactionID of MetaD node MDj is the transaction ID of Tj, and the BlockID of MetaD node MDj is the block ID of Tj.
[0045] Step 3.1.5: In the Gskiplist subsystem, sequentially search the level 0 "Node tree" to find a Node Nm whose key value range includes Kj. The key value range of this Node Nm is denoted as [x, y), where x <= y; when x equals y, the key value range of Nm only represents x; initially, the level 0 "Node tree" has only one Node node, whose key value range is [-∞, +∞).
[0046] Step 3.1.6: Insert the MetaD node MDj information from step 3.1.4 into the "MetaD node set" of the Node node Nm from step 3.1.5;
[0047] Step 3.1.7: Calculate the number of MetaD nodes owned by the Node Nm in step 3.1.5. If the number of MetaD nodes owned by the Node Nm reaches the "MetaD node" fission threshold S read in step 2, then a value t can be calculated so that the Node Nm can be fissioned into two new Nodes. The key value ranges of the two new Nodes are [x,t) and [t,y), respectively. The principle for choosing the value t is to make the two Nodes have approximately the same number of MetaD nodes.
[0048] Step 3.1.8: Determine whether all transactions in block Bi have been traversed. If all transactions in block Bi have been traversed, proceed to step 3.1.9 to continue execution; otherwise, proceed to step 3.1.2 to process the next transaction.
[0049] Step 3.1.9: Determine whether all blocks in the blockchain have been traversed. If all blocks in the blockchain have been traversed, proceed to step 3.1.10 to continue execution; otherwise, proceed to step 3.1.1 to process the next block.
[0050] Step 3.1.10: Processing ends.
[0051] Furthermore, according to the access method provided by this invention, the construction of the initial preset "Node tree" for levels 1 to K of the Gskiplist subsystem in step 3.2 is performed by upgrading some "Node nodes" of the level 0 "Node tree" according to the initial preset level Q and the initial preset rule type R of the "Node tree" in the "Gskiplist parameter file", thereby gradually building the "Node tree" for levels 1 to K. The initial preset rule type R of the "Node tree" can include the following rule types: binary rule, ternary rule, N-part rule, and custom rule. The binary rule refers to upgrading one of every two consecutive nodes in the same level of the node tree; the trichotomy rule refers to upgrading one of every three consecutive nodes in the same level of the node tree; the N-partition rule refers to upgrading one of every N consecutive nodes in the same level of the node tree; and the custom rule refers to selecting several nodes in the same level of the node tree according to the designer's custom rules for upgrading.
[0052] Furthermore, according to the access method provided by this invention, the dynamic adjustment of "Nodes" in step 14.1 of the "dynamic structural adjustment" of the Gskiplist subsystem is based on whether the "retrieval count, i.e., SCount" of each "Node" meets the upgrade threshold U condition of the "Node" in the "Gskiplist parameter file". If the upgrade condition is met, the "Node" is promoted one level in the "Node tree". The steps for promoting a "Node" one level are as follows:
[0053] Step 14.1.u.1: Obtain the highest level of the "Node" to be upgraded, denoted as g, and upgrade the "Node" to level g+1.
[0054] Step 14.1.u.2: In the g level of the "Node Tree", obtain the left-side link pointer array Prev[g] of the "Node" to be upgraded and obtain the left-side adjacent "Node"; if the left-side adjacent "Node" does not have a g+1 level, continue to obtain the left-side adjacent "Node" of the left-side adjacent "Node" until a left-side adjacent "Node" with a g+1 level is obtained;
[0055] Step 14.1.u.3: In the g level of the "Node Tree", obtain the right-side link pointer array Next[g] of the "Node" to be upgraded and obtain the right-side adjacent "Node"; if the right-side adjacent "Node" does not have a g+1 level, continue to obtain the right-side adjacent "Node" of the right-side adjacent "Node" until a right-side adjacent "Node" with a g+1 level is obtained.
[0056] Step 14.1.u.4: Set the right link pointer array Next[g+1] of the left adjacent "Node" obtained in step 14.1.u.2 to the "Node" to be upgraded; further set the left link pointer array Prev[g+1] of the right adjacent "Node" obtained in step 14.1.u.3 to the "Node" to be upgraded.
[0057] Step 14.1.u.5: Further set the left link pointer array Prev[g+1] of the "Node" to be upgraded to the left adjacent "Node" obtained in step 14.1.u.2, and set the right link pointer array Next[g+1] of the "Node" to be upgraded to the right adjacent "Node" obtained in step 14.1.u.3;
[0058] Step 14.1.u.6: Upgrade the "Node" to be upgraded to level g+1. The upgrade process is now complete.
[0059] Furthermore, according to the access method provided by this invention, the dynamic adjustment of the "Node" in step 14.1 of the "dynamic structural adjustment" of the Gskiplist subsystem is based on whether the "retrieval count, i.e., SCount" of each "Node" meets the "Node" downgrade threshold D condition in the "Gskiplist parameter file". If the downgrade condition is met, the "Node" is downgraded one level in the "Node tree". The steps of the "Node" downgrade method are as follows:
[0060] Step 14.1.d.1: Obtain the highest level of the "Node" to be downgraded, denoted as g, and intend to downgrade the "Node" to level g-1;
[0061] Step 14.1.d.2: In the g level of the "Node Tree", obtain the left-side link pointer array Prev[g] of the "Node" to be demoted and obtain the left-side adjacent "Node".
[0062] Step 14.1.d.3: In the g level of the "Node Tree", obtain the right-side link pointer array Next[g] of the "Node" to be demoted and obtain the right-side adjacent "Node".
[0063] Step 14.1.d.4: Set the right link pointer array Next[g] of the left adjacent "Node" obtained in step 14.1.d.2 to the right adjacent "Node" obtained in step 14.1.d.3; further set the left link pointer array Prev[g] of the right adjacent "Node" obtained in step 14.1.d.3 to the left adjacent "Node" obtained in step 14.1.d.2;
[0064] Step 14.1.d.5: Set both Prev[g] and Next[g] of the "Node" to be downgraded to empty;
[0065] Step 14.1.d.6: Downgrade the "Node" to be upgraded to level g-1. The downgrade process is now complete.
[0066] Furthermore, according to the access method provided by this invention, the dynamic adjustment of the splitting of the "MetaD node set" in step 14.1 of the "dynamic structural adjustment" of the Gskiplist subsystem is for a given "Node". If the number of "MetaD nodes" in the "MetaD node set" pointed to by the MetaDID of the "Node" is greater than or equal to the "MetaD node" splitting threshold S in the "Gskiplist parameter file", then the given "Node" becomes the "Node" to be split, which can be split into two new "Node nodes" with non-overlapping key value ranges. Based on the key value ranges of the two new "Node nodes", the original "MetaD node set" is further split into two new "MetaD node sets", each new "Node node" pointing to a new "MetaD node set". Furthermore, the new "Node node" with the key value range on the left inherits the link relationship of the original given "Node node" in the "Node node tree". The steps of the method for splitting the "MetaD node set" are as follows:
[0067] Step 14.1.s.1: Obtain the key value range [KeyLeft, KeyRight] of the "Node" to be split, and obtain the "MetaD node set" pointed to by the MetaDID in the "Node" to be split;
[0068] Step 14.1.s.2: Sort the “MetaD node set” in step 14.1.s.1 by the key of the “MetaD node”, and select the median value KeyM of the key.
[0069] Step 14.1.s.3: Based on KeyM in step 14.1.s.2, divide the "Node" to be split into two new "Nodes" with key value ranges of [KeyLeft, KeyM) and [KeyM, KeyRight), respectively. The new "Node" on the left inherits the link relationship of the original "Node" in the "Node tree", and the new "Node" on the right is a completely new "Node".
[0070] Step 14.1.s.4: Traverse the "MetaD Node Set" pointed to by the new "Node" on the left in step 14.1.s.3, and migrate all "MetaD Nodes" whose key is not in the key value range [KeyLeft, KeyM) of the new "Node" on the left to the "MetaD Node Set" pointed to by the new "Node" on the right in step 14.1.s.3;
[0071] Step 14.1.s.5: Complete the splitting of the "MetaD node set". The splitting process is now complete.
[0072] Furthermore, according to the access method provided by this invention, the dynamic adjustment of the "MetaD node set" in step 14.1 of the "dynamic structural adjustment" of the Gskiplist subsystem is for two adjacent "Node nodes" in the level 0 "Node tree". If the number of "MetaD nodes" in the "MetaD node set" pointed to by the MetaDID of these two "Node nodes" is less than the "MetaD node" merging threshold M in the "Gskiplist parameter file", then the two "Node nodes" are merged into a new "Node node", and the corresponding two "MetaD node sets" are also merged together. Furthermore, the link relationship of these two "Node nodes" in the "Node tree" is inherited by the new "Node node". The specific method is to first obtain the highest level of each of the two "Node nodes" in the "Node tree", and then select the "Node node" with the highest level as the new "Node node". If the levels are the same, one "Node node" is selected as the new "Node node", and the link relationship of the other original "Node node" in the "Node tree" is further merged into the new "Node node".
[0073] Furthermore, according to the access method provided by this invention, the specific steps of the "persistent storage decision" in step 14.2 of the "persistent storage decision and implementation" of the Gskiplist subsystem are as follows:
[0074] Step 14.2.1: First, obtain the actual memory space occupied by the Gskiplist subsystem;
[0075] Step 14.2.2: Then calculate the ratio r of the actual memory space occupied by the Gskiplist subsystem to the "maximum memory space occupied G" in the "Gskiplist parameter file";
[0076] Step 14.2.3: Determine whether the ratio r in step 14.2.2 is greater than or equal to the "persistent storage decision threshold P" in the "Gskiplist parameter file". If it is greater than or equal to, proceed with "persistent storage implementation"; otherwise, do not proceed with "persistent storage implementation".
[0077] Step 14.2.4, “Persistent storage decision” processing ends.
[0078] Furthermore, according to the access method provided by this invention, in step 14.2, the "persistent storage implementation" of the "persistent storage decision and implementation" of the Gskiplist subsystem, this invention uses a database as the carrier of persistent storage. The database can be a key-value database or a relational database. The database storage format can be a key-value format, i.e., (key, value), where key is the key and value is the value corresponding to the key. In the "Node", the key is NodeID, and the value is the "recursive length prefix encoding" (RLP encoding) of other fields. The value is the result of RLP encoding of the set of all non-NodeID fields in the "Node". In the "MetaD", the key is MetaDID, and the value is the result of RLP encoding of the set of all non-MetaDID fields in the "MetaD".
[0079] The specific content involved in this invention and the meaning of the terms used are as follows:
[0080] (1) Skip list: William Pugh published Skip lists: a probabilistic alternative to balanced trees in Communications of the ACM June 1990, 33(6) 668-676. Skip list was originally designed as an alternative to balanced trees.
[0081] (2) Grouped Skiplist (Gskiplist): This is a novel skip list proposed in this invention. A grouped skip list is a skip list that supports a set of key value ranges, and its structure can be dynamically adjusted according to the "user's query requirements".
[0082] (3) Blockchain: It is a decentralized and tamper-proof distributed database that is widely used in finance, Internet of Things, public services, notarization, digital copyright, traceability and other fields.
[0083] (4) Blockchain query system: This refers to a system that provides query and statistical services based on publicly available and tamper-proof data on the blockchain. With the increasing demands of actual business operations, blockchain query systems are required to provide reliable, authoritative, and efficient complex queries, such as Top-k queries, range queries, and equality queries.
[0084] (5) RLP: A data encoding rule, short for Recursive Length Prefix, which can be used to encode arbitrarily nested binary array data. The result of RLP encoding is also a binary sequence. RLP can effectively serialize and deserialize data.
[0085] Advantages and positive effects of the present invention:
[0086] This invention can effectively improve the efficiency of blockchain query and retrieval.
[0087] The efficient blockchain query system and dynamic construction method based on Gskiplist provided by this invention achieve efficient querying and retrieval of blockchain data without affecting the decentralized nature of blockchain and the original data storage method. This invention ensures both the immutability, openness, and authority of blockchain data, and guarantees the efficiency of blockchain data querying and retrieval, providing efficient query services for an increasing number of blockchain-based applications while ensuring the availability of a single set of blockchain data. Attached Figure Description
[0088] Figure 1 This is a schematic diagram of the system architecture of the efficient blockchain query system based on Gskiplist in this invention.
[0089] Figure 2 This is a schematic diagram of the basic structure of transactions in the blockchain of this invention.
[0090] Figure 3 This is a schematic diagram of the basic structure of a block in the blockchain of this invention.
[0091] Figure 4 This is a schematic diagram of the basic structure of the "Node" in the Gskiplist subsystem of this invention.
[0092] Figure 5 This is a schematic diagram of the basic structure of the "MetaD node" in the Gskiplist subsystem of this invention.
[0093] Figure 6 This is a schematic diagram of the basic structure of the Gskiplist parameter file in the Gskiplist subsystem of this invention.
[0094] Figure 7 This is a schematic diagram of the structure of the "Node tree" in the Gskiplist subsystem before the upgrade in this invention.
[0095] Figure 8 This is a schematic diagram of the upgraded "Node tree" structure in the Gskiplist subsystem of this invention. Detailed Implementation
[0096] This invention discloses a high-efficiency blockchain query system based on Gskiplist and a dynamic construction method. For example... Figure 1As shown, the efficient blockchain query system based on Gskiplist provided by this invention includes three parts: a blockchain subsystem 110, a "grouped skip list" subsystem 120, and a database 100. The blockchain subsystem 110 contains all "transactions" 112 and "blocks" 111. Each "transaction" 111 contains at least a transaction ID (TransactionID), transaction time, transaction participants, transaction header hash, and transaction content information. Each "block" 112 contains transactions within a given time period. Blocks are linked together in a chain based on their creation time. Each "block" contains at least a block ID (BlockID), block number, block start and end time, the previous block ID, the Merkle root of the contained "transaction" set, the block header hash, and the contained "transaction" set information. All "transactions" 112 and "blocks" 111 data in the blockchain subsystem 110 are stored in the database.The "Grouped Skip List" subsystem 120, abbreviated as Gskiplist subsystem, contains a multi-level "Node Tree" 121 composed of "Nodes" and multiple "MetaD Node Sets" 122 composed of "MetaD Nodes". Each "Node" contains at least the Node ID (NodeID), the left key (KeyLeft), the right key (KeyRight), the ID of the first MetaD node (MetaDID), the search count (SCount), the pointer array Prev of the previous "Node", and the pointer array of the next "Node". The Next array contains information about each "Node". Each "Node" contains a set of "MetaD Nodes" within the range [KeyLeft, KeyRight) that are greater than or equal to the left key and less than the right key. Each "Node" also forms a multi-level chain with other "Nodes" through pointer arrays Prev and Next. Each "MetaD Node" contains at least the "MetaD Node" ID (MetaDID), the key (Key), the corresponding "Node" ID, the corresponding blockchain transaction ID (TransactionID), and the corresponding blockchain transaction ID. The block ID is BlockID, the MetaDID of the previous MetaD node is Prev, and the MetaDID of the next MetaD node is Next. The Gskiplist subsystem also includes a Gskiplist parameter file to define the multi-level Node tree. The Gskiplist parameter file must contain at least the Gskiplist ID (GSL_ID), the Gskiplist search field set (GSL_SearchFields), the maximum level N of the Node tree, the initial preset level Q of the Node tree, and so on. The initial preset rule type R of the "node tree", the upgrade threshold U of the "node", the downgrade threshold D of the "node", the fission threshold S of the "metaD node", the merging threshold M of the "metaD node", the maximum memory space occupied value G, and the persistent storage decision threshold P are all stored in the database. All "node" and "metaD node" data in the Gskiplist subsystem are stored in the database. Database 100 is used to store the transaction data 112 and block data 111 of the blockchain subsystem 110, and the database is also used to store the "node" and "metaD node" data of the Gskiplist subsystem 120.
[0097] The specific steps of the dynamic construction method for the efficient blockchain query system based on Gskiplist provided by this invention are as follows:
[0098] Step 1: Start the system and set the "Gskiplist parameter file" to be read;
[0099] Step 2: Read the "Gskiplist parameter file" to obtain the Gskiplist ID (GSL_ID), the Gskiplist search field set (GSL_SearchFields), the maximum level of the "Node tree" (N), the initial preset level of the "Node tree" (Q), the initial preset rule type of the "Node tree" (R), the "Node" upgrade threshold (U), the "Node" downgrade threshold (D), the "MetaD" splitting threshold (S), the "MetaD" merging threshold (M), the maximum memory space occupied (G), and the persistent storage decision threshold (P).
[0100] Step 3: Read the blockchain data and build the initial preset "group skip list" subsystem. The specific steps are as follows:
[0101] Step 3.1: Based on the GSL_SearchFields set of search fields in step 2, read the blockchain data to construct the level 0 "Node tree" and multiple "MetaD node sets";
[0102] Step 3.2: Based on the initial preset level Q of the "Node Tree" in Step 2, construct the initial preset "Node Tree" from level 1 to level Q on the basis of the level 0 Node Tree;
[0103] Step 4: The system enables blockchain query services to be provided to external parties;
[0104] Step 5: Monitor the generation of new blocks in the blockchain subsystem, and simultaneously monitor and receive "query requests from the user".
[0105] Step 6: Determine the type of the listening response. If it is "a new block generated in the blockchain subsystem", proceed to step 17; if it is "a query request from the user", proceed to step 7.
[0106] Step 7: Parse the user's query request;
[0107] Step 8: Convert the query request in step 7 into the key value range [Vsb, Vse) to be queried;
[0108] Step 9: Search step by step from the "highest level" of the Gskiplist subsystem down to level 0, finding all "Nodes" corresponding to the key value range [Vsb, Vse) given in step 8. The "highest level" of the Gskiplist subsystem starts at K, but changes dynamically as the structure of the Gskiplist subsystem changes. The [KeyLeft, KeyRight) of each retrieved "Node" must have a non-empty intersection with the key value range [Vsb, Vse) given in step 8.
[0109] Step 10: Among all the “Nodes” found in step 9, further search for all “MetaD Nodes” corresponding to the key value range [Vsb, Vse) given in step 8. That is, the key of the retrieved “MetaD Node” must be within the key value range [Vsb, Vse) given in step 8.
[0110] Step 11: Based on the transaction ID (TransactionID) and block ID (BlockID) information of all the "MetaD nodes" retrieved in step 10, further search for the relevant "blockchain data" in the blockchain.
[0111] Step 12: For all the "blockchain data" found in step 11, further calculate the "final query result" of the "user's query request";
[0112] Step 13: For all the "Nodes" found in step 9, increment the "retrieval count (SCount)" of each "Node" by 1;
[0113] Step 14: Dynamically adjust the structure of the Gskiplist subsystem using synchronous or asynchronous methods. The specific steps are as follows:
[0114] Step 14.1: Perform "dynamic structural adjustment" on the Gskiplist subsystem using synchronous or asynchronous methods, that is, dynamically adjust the upgrade or downgrade of "Node nodes" and dynamically adjust the splitting or merging of "MetaD node sets";
[0115] Step 14.2: Perform "persistent storage decision and implementation" on the Gskiplist subsystem using synchronous or asynchronous methods, and save all in-memory data of the Gskiplist subsystem to the database;
[0116] Step 15: Return the "final query result" calculated in step 12 to the user.
[0117] Step 16: Determine whether the system should continue running. If the system continues running, proceed to step 5; if the system does not continue running, proceed to step 18.
[0118] Step 17: Read the "new block" of the blockchain and perform "content update" on the Gskiplist subsystem, that is, update multiple "MetaD node sets" and "Node node trees" according to the transaction information in the "new block"; then jump to step 5 to continue execution;
[0119] Step 18: Implement persistent storage for the Gskiplist subsystem using a synchronous method, saving all in-memory data of the Gskiplist subsystem to the database;
[0120] Step 19: System execution complete.
[0121] To further understand the technical content, features, and effects of this invention, the following application examples are provided, along with detailed descriptions in conjunction with the accompanying drawings:
[0122] Application Example 1
[0123] The specific steps of a dynamic construction method for an efficient blockchain query system based on Gskiplist are as follows:
[0124] Step 1: Start the system and set the "Gskiplist parameter file" to be read;
[0125] For example: Set the "Gskiplist parameter file" to the gskiplist_gas_config.yaml file.
[0126] Step 2: Read the "Gskiplist parameter file" to obtain the Gskiplist ID (GSL_ID), the Gskiplist search field set (GSL_SearchFields), the maximum level of the "Node tree" (N), the initial preset level of the "Node tree" (Q), the initial preset rule type of the "Node tree" (R), the "Node" upgrade threshold (U), the "Node" downgrade threshold (D), the "MetaD" splitting threshold (S), the "MetaD" merging threshold (M), the maximum memory space occupied (G), and the persistent storage decision threshold (P).
[0127] For example:
[0128] The `gskiplist_gas_config.yaml` file contains the following parameters: `GSL_ID` is 1, `GSL_SearchFields` is "Gas", the maximum level of the "Node" tree (N) is 20, the initial preset level of the "Node" tree (Q) is 14, the initial preset rule type of the "Node" tree (R) is the default value of 1, the "Node" upgrade threshold (U) is 10, the "Node" downgrade threshold (D) is 3, the "MetaD" node fission threshold (S) is 10, the "MetaD" node merge threshold (M) is 3, the maximum memory space occupied (G) is 128 MB, and the persistent storage decision threshold (P) is 0.7.
[0129] The "github.com / spf13 / viper" component can be used to manage memory configurations. Call the Viper component's ReadInConfig() method, pass in the absolute path of the parameter file on the disk, read the configuration saved in the parameter file, and then call the Viper component's Unmarshal() method to deserialize the configuration into a memory object gskiplist_gas_config.
[0130] Step 3: Read the blockchain data and build the initial preset "group skip list" subsystem. The specific steps are as follows:
[0131] Step 3.1: Based on the GSL_SearchFields set of search fields in step 2, read the blockchain data to construct the level 0 "Node tree" and multiple "MetaD node sets";
[0132] For example:
[0133] First, install the plugin github.com / ethereum / go-ethereum. Then, call the ethclient.Dial() method and pass in the Ethereum RPC call address to successfully create an Ethereum client object to interact with Ethereum. Next, use the method global.EthRpcClient.BlockByNumber() provided by the client object and pass in the blocknumber parameter to get all the information of the block. Then, iterate through the transaction array in the block object to get all the transaction receipt objects in the block, which store all the transaction information.
[0134] Blockchain transaction structure such as Figure 2 As shown, the block structure of a blockchain is as follows: Figure 3 As shown.
[0135] Then, it reads GSL_SearchFields to determine that the Gskiplist created this time is sorted by the transaction's Gas value as the key. Therefore, it reads the data from the Ethereum transaction receipt object and creates the corresponding MetaD object, such as... Figure 5 As shown.
[0136] like Figure 5 As shown, MetaDID is a globally unique auto-incrementing ID of the MetaD object. The Key corresponds to the Gas value of this transaction, which is the EffectiveGasPrice in the Receipt. TransactionID corresponds to TxHash in the Receipt, and BlockID corresponds to BlockNumber in the Receipt.
[0137] Then, the `Search()` method of `Gskiplist` is called, passing in the key of the `MetaD`. It finds a `Node` in `Gskiplist` whose `[KeyLeft, KeyRight)` contains the key of the `MetaD`. The `MetaD` is then linked to the `MetaD` set of the corresponding `Node` object. If the `Node` does not exist, a new `Node` object is created. Figure 4 As shown.
[0138] like Figure 4 As shown, NodeID is a globally unique auto-incrementing ID of the Node object, KeyLeft represents the left boundary of the key range of this Node, KeyRight represents the right boundary of the key range of this Node, MetaDID represents the first MetaDID of the MetaD set corresponding to this Node, the Prev array stores the previous NodeID of this Node at each level in Gskiplist, and the Next array stores the next NodeID of this Node at each level in Gskiplist.
[0139] After successful creation, the Pre_Add() method of Gskiplist is called. This method will find the linked list position of the key of the new Node in the key-ordered level 0 "Node tree", insert the new Node into that position in the level 0 "Node tree", and link the new MetaD to the MetaD set of this new Node.
[0140] Step 3.2: Based on the initial preset level Q of the "Node Tree" in Step 2, construct the initial preset "Node Tree" from level 1 to level Q on the basis of the level 0 Node Tree;
[0141] For example:
[0142] Read the variable Q in the gskiplist_gas_config object to get the initial preset level as 14. We designed the initial ideal model of Gskiplist so that the number of nodes in each level is 1 / 2 of the next level. Therefore, we use the conditional probability algorithm to calculate that each Node should exist in levels 0 to N of Gskiplist (N≤Q). We assume that the probability of N plus 1 is 1 / 2. We calculate the size of N and then build the Node into a "Node tree" from level 1 to N. Specifically, we edit the Prev and Next arrays of the Node. For example, if we build the Node into the first level "Node tree", we traverse all the Nodes in the first level "Node tree" and find the position where the new Node should be stored according to the Node's Key. That is, the KeyRight of the previous Node ≤ the KeyLeft of the new Node and the KeyLeft of the next Node ≥ the KeyRight of the new Node. Then we modify the [1]Next of the previous Node and the [1]Prev of the next Node, and update the [1]Next and [1]Prev of the new Node. We then build the "Node tree" from level 1 to N.
[0143] Step 4: The system enables blockchain query services to be provided to external parties;
[0144] For example:
[0145] Modify the flag in the memory configuration to 1 to indicate that blockchain query service is provided. Each query request will first access the flag, and the query can only be executed if the flag is 1.
[0146] Step 5: Monitor the generation of new blocks in the blockchain subsystem, and simultaneously monitor and receive "query requests from the user".
[0147] For example:
[0148] Use the `time` package in Go to start a scheduled task to periodically check the BlockNumber of the latest Ethereum block, and at the same time establish a TCP connection with the client based on socket programming to receive query requests from the client.
[0149] Step 6: Determine the type of the listening response. If it is "a new block generated in the blockchain subsystem", proceed to step 17; if it is "a query request from the user", proceed to step 7.
[0150] Step 7: Parse the user's query request;
[0151] For example:
[0152] Based on the flag bits of the user's request parameters, determine the user's query type, which may be a key equality query, a key range query, or a top-K key query.
[0153] Step 8: Convert the query request in step 7 into the key value range [Vsb, Vse) to be queried;
[0154] For example:
[0155] The key value range [Vsb, Vse) for querying blockchain gas fees is [0.01, 0.05].
[0156] Step 9: Search step by step from the "highest level" of the Gskiplist subsystem down to level 0, finding all "Nodes" corresponding to the key value range [Vsb, Vse) given in step 8. The "highest level" of the Gskiplist subsystem starts at K, but changes dynamically as the structure of the Gskiplist subsystem changes. The [KeyLeft, KeyRight) of each retrieved "Node" must have a non-empty intersection with the key value range [Vsb, Vse) given in step 8.
[0157] For example:
[0158] The [KeyLeft, KeyRight) of each retrieved "Node" should have a non-empty intersection with the example [0.01, 0.05) in step 8, that is, the key value of [KeyLeft, KeyRight) needs to contain or partially contain [0.01, 0.05).
[0159] Step 10: Among all the “Nodes” found in step 9, further search for all “MetaD Nodes” corresponding to the key value range [Vsb, Vse) given in step 8. That is, the key of the retrieved “MetaD Node” must be within the key value range [Vsb, Vse) given in step 8.
[0160] Step 11: Based on the transaction ID (TransactionID) and block ID (BlockID) information of all the "MetaD nodes" retrieved in step 10, further search for the relevant "blockchain data" in the blockchain.
[0161] For example:
[0162] The transaction information and block information of the blockchain are as follows: Figure 2 , Figure 3 As shown.
[0163] Step 12: For all the "blockchain data" found in step 11, further calculate the "final query result" of the "user's query request";
[0164] For example:
[0165] Depending on the different query types, such as key equality query, key range query, and key Top-K query, the found "blockchain data" is further calculated by accumulating or sorting, and finally the "final query result" of the "user's query request" is generated.
[0166] Step 13: For all the "Nodes" found in step 9, increment the "retrieval count (SCount)" of each "Node" by 1;
[0167] For example:
[0168] Increment the "retrieval count (SCount)" of each "Node" by 1 to prepare for further dynamic adjustments to the "Node tree".
[0169] Step 14: Dynamically adjust the structure of the Gskiplist subsystem using synchronous or asynchronous methods. The specific steps are as follows:
[0170] Step 14.1: Perform "dynamic structural adjustment" on the Gskiplist subsystem using synchronous or asynchronous methods, that is, dynamically adjust the upgrade or downgrade of "Node nodes" and dynamically adjust the splitting or merging of "MetaD node sets";
[0171] For example:
[0172] Frequently accessed "Nodes" may be upgraded, thereby speeding up retrieval.
[0173] Infrequently accessed "Nodes" may be downgraded to speed up retrieval and minimize the memory size of Gskiplist.
[0174] For a large set of "MetaD nodes", it may be split to reduce the cost of searching within the "MetaD node set";
[0175] For very small sets of "MetaD nodes", they may be merged to reduce the number of "MetaD nodes", which also helps to minimize the memory size of Gskiplist;
[0176] Step 14.2: Perform "persistent storage decision and implementation" on the Gskiplist subsystem using synchronous or asynchronous methods, and save all in-memory data of the Gskiplist subsystem to the database;
[0177] For example:
[0178] The Gskiplist subsystem determines whether to immediately implement "persistent storage" (i.e., "persistent storage implementation") based on the current memory usage of Gskiplist, thereby ensuring both the operational efficiency and data security of the Gskiplist subsystem.
[0179] Step 15: Return the "final query result" calculated in step 12 to the user.
[0180] Step 16: Determine whether the system should continue running. If the system continues running, proceed to step 5; if the system does not continue running, proceed to step 18.
[0181] Step 17: Read the "new block" of the blockchain and perform "content update" on the Gskiplist subsystem, that is, update multiple "MetaD node sets" and "Node node trees" according to the transaction information in the "new block"; then jump to step 5 to continue execution;
[0182] For example:
[0183] The `EthRpcClient.BlockNumber()` method of the `go-ethereum` plugin retrieves the latest BlockNumber in the current Ethereum blockchain. This BlockNumber is then compared to the `LastBlockNumber` stored in memory. If they match, the data in the `Gskiplist` is up-to-date. If the former is larger, the `Gskiplist` needs to be updated with all blocks between `LastBlockNumber` and the latest BlockNumber. The specific update steps are similar to step 3.1: all transaction receipts for the block are obtained through the Ethereum client object. A corresponding MetaD is created based on the transaction receipt content, and then the MetaD is linked to the corresponding Node. If no corresponding Node exists, a new Node is created and inserted into the `Gskiplist`. A conditional probability algorithm is used to calculate the appropriate level (1 to N) in the `Gskiplist` to insert the Node into. Then, the process jumps to step 5 to continue execution.
[0184] Step 18: Implement persistent storage for the Gskiplist subsystem using a synchronous method, saving all in-memory data of the Gskiplist subsystem to the database;
[0185] For example:
[0186] Before the system shuts down, it's necessary to implement persistent storage for the Gskiplist subsystem, which means persisting the updated Nodes and MetaDs in memory to a database. Ethereum implements Leveldb as its underlying storage engine for persisting all Ethereum data. We can use the Batch() method of the go-ethereum / ethdb / leveldb plugin to batch execute the Put() method, persisting Nodes and MetaDs to Leveldb and ensuring the decentralized nature of the Gskiplist index data is not lost.
[0187] Step 19: System execution complete.
[0188] Furthermore, the construction of the level 0 "Node tree" and multiple "MetaD node sets" for the Gskiplist subsystem described in step 3.1 involves sequentially reading blockchain blocks and transaction data, and establishing the level 0 "Node tree" and multiple "MetaD node sets" based on the Gskiplist search field set, i.e., GSL_SearchFields, from step 2. This can be achieved using the following method:
[0189] Step 3.1.1: Read the block data from block 0 to the "maximum ID block" in the blockchain sequentially. Each block read is abbreviated as Bi, where 0 <= i < "maximum ID block". Perform the following processing on each block Bi;
[0190] For example:
[0191] Block information in the blockchain, such as Figure 3 As shown.
[0192] Step 3.1.2: Traverse each transaction in block Bi, denoted as Tj, where 0 <= j < "the maximum number of transactions in block Bi". Perform the following processing on each transaction Tj;
[0193] For example:
[0194] Blockchain transaction information such as Figure 2 As shown.
[0195] Step 3.1.3: Calculate the key and value (key, value) for transaction Tj based on the set of search fields of Gskiplist in step 2, namely GSL_SearchFields, and record the result as (Kj, Vj).
[0196] For example:
[0197] Gas fees are used as the search field set.
[0198] Step 3.1.4: Create a MetaD node MDj for transaction Tj. The key value of MetaD node MDj is Kj, the TransactionID of MetaD node MDj is the transaction ID of Tj, and the BlockID of MetaD node MDj is the block ID of Tj.
[0199] For example:
[0200] like Figure 5 The structure of the MetaD node is shown below.
[0201] Step 3.1.5: In the Gskiplist subsystem, sequentially search the level 0 "Node tree" to find a Node Nm whose key value range includes Kj. The key value range of this Node Nm is denoted as [x, y), where x <= y; when x equals y, the key value range of Nm only represents x; initially, the level 0 "Node tree" has only one Node node, whose key value range is [-∞, +∞).
[0202] For example:
[0203] like Figure 4 The diagram shows the structure of a Node.
[0204] Initially, the level 0 "Node tree" has only one Node node with a key value range of [-∞, +∞); then through the "split" operation, a Node node will be dynamically split into several Node nodes.
[0205] Step 3.1.6: Insert the MetaD node MDj information from step 3.1.4 into the "MetaD node set" of the Node node Nm from step 3.1.5;
[0206] For example:
[0207] The MetaDID of a “Node” points to the first “MetaD node” in a “MetaD node set”, and other “MetaD nodes” are found based on the Next and Prev of this “MetaD node”.
[0208] Step 3.1.7: Calculate the number of MetaD nodes owned by the Node Nm in step 3.1.5. If the number of MetaD nodes owned by the Node Nm reaches the "MetaD node" fission threshold S read in step 2, then a value t can be calculated so that the Node Nm can be fissioned into two new Nodes. The key value ranges of the two new Nodes are [x,t) and [t,y), respectively. The principle for choosing the value t is to make the two Nodes have approximately the same number of MetaD nodes.
[0209] For example:
[0210] This step enables the splitting or fission of "MetaD nodes" and their corresponding "Node nodes", thereby dynamically increasing the number of "Node nodes" and "MetaD nodes".
[0211] Step 3.1.8: Determine whether all transactions in block Bi have been traversed. If all transactions in block Bi have been traversed, proceed to step 3.1.9 to continue execution; otherwise, proceed to step 3.1.2 to process the next transaction.
[0212] Step 3.1.9: Determine whether all blocks in the blockchain have been traversed. If all blocks in the blockchain have been traversed, proceed to step 3.1.10 to continue execution; otherwise, proceed to step 3.1.1 to process the next block.
[0213] Step 3.1.10: Processing ends.
[0214] Furthermore, the construction of the initial preset "Node Tree" for levels 1 to K in step 3.2 of the Gskiplist subsystem is carried out by upgrading some "Nodes" in the level 0 "Node Tree" according to the initial preset level Q and the initial preset rule type R of the "Node Tree" in the "Gskiplist parameter file", thereby gradually building the "Node Tree" for levels 1 to K. The initial preset rule type R of the "Node Tree" can include the following rule types: binary rule, triadic rule, N-participation rule, and custom rule. The binary rule refers to upgrading one "Node" in every two consecutive "Nodes" in the same level of the "Node Tree". The triadic rule refers to upgrading one "Node" in every three consecutive "Nodes" in the same level of the "Node Tree". The N-participation rule refers to upgrading one "Node" in every N consecutive "Nodes" in the same level of the "Node Tree". The custom rule refers to selecting several "Nodes" in the same level of the "Node Tree" according to the rules defined by the designer for upgrading.
[0215] For example:
[0216] For the level 0 "Node tree", a binary search rule is used. That is, in the level 0 "Node tree", one of every two consecutive "Nodes" is upgraded to become the level 1 "Node tree". Therefore, the level 1 "Node tree" will have half of the "Nodes" in the level 0 "Node tree". In this way, the search does not need to traverse every level 0 "Node tree", thus speeding up the search efficiency.
[0217] Furthermore, the "dynamic structural adjustment" of the Gskiplist subsystem described in step 14.1, specifically the dynamic adjustment of "Nodes," is based on whether the "retrieval count (SCount)" of each "Node" meets the upgrade threshold U condition in the "Gskiplist parameter file." If the upgrade condition is met, the "Node" is promoted one level in the "Node tree." The steps for promoting a "Node" are as follows:
[0218] Step 14.1.u.1: Obtain the highest level of the "Node" to be upgraded, denoted as g, and upgrade the "Node" to level g+1.
[0219] For example:
[0220] like Figure 7As shown, upgrading the "Node" to be upgraded from level g to level g+1 yields the following result: Figure 8 The results are shown.
[0221] Step 14.1.u.2: In the g level of the "Node Tree", obtain the left-side link pointer array Prev[g] of the "Node" to be upgraded and obtain the left-side adjacent "Node"; if the left-side adjacent "Node" does not have a g+1 level, continue to obtain the left-side adjacent "Node" of the left-side adjacent "Node" until a left-side adjacent "Node" with a g+1 level is obtained;
[0222] Step 14.1.u.3: In the g level of the "Node Tree", obtain the right-side link pointer array Next[g] of the "Node" to be upgraded and obtain the right-side adjacent "Node"; if the right-side adjacent "Node" does not have a g+1 level, continue to obtain the right-side adjacent "Node" of the right-side adjacent "Node" until a right-side adjacent "Node" with a g+1 level is obtained.
[0223] For example:
[0224] Before upgrading a "Node", it is necessary to find the left and right adjacent "Nodes", and the left and right adjacent "Nodes" also need to have a level of g+1. Therefore, steps 14.1.u.2 and 14.1.u.3 are used to find the left and right adjacent "Nodes" that meet the requirements.
[0225] Step 14.1.u.4: Set the right link pointer array Next[g+1] of the left adjacent "Node" obtained in step 14.1.u.2 to the "Node" to be upgraded; further set the left link pointer array Prev[g+1] of the right adjacent "Node" obtained in step 14.1.u.3 to the "Node" to be upgraded.
[0226] Step 14.1.u.5: Further set the left link pointer array Prev[g+1] of the "Node" to be upgraded to the left adjacent "Node" obtained in step 14.1.u.2, and set the right link pointer array Next[g+1] of the "Node" to be upgraded to the right adjacent "Node" obtained in step 14.1.u.3;
[0227] For example:
[0228] In upgrading a "Node", it is necessary to implement the connection and interconnection of the left and right adjacent "Nodes" at level g+1.
[0229] Step 14.1.u.6: Upgrade the "Node" to be upgraded to level g+1. The upgrade process is now complete.
[0230] Furthermore, the dynamic adjustment of "Node" in the "dynamic structure adjustment" of the Gskiplist subsystem described in step 14.1 is based on whether the "retrieval count (SCount)" of each "Node" meets the "Node" degradation threshold D condition in the "Gskiplist parameter file". If the degradation condition is met, the "Node" is demoted one level in the "Node tree". The steps for demoting a "Node" by one level are as follows:
[0231] Step 14.1.d.1: Obtain the highest level of the "Node" to be downgraded, denoted as g, and intend to downgrade the "Node" to level g-1;
[0232] Step 14.1.d.2: In the g level of the "Node Tree", obtain the left-side link pointer array Prev[g] of the "Node" to be demoted and obtain the left-side adjacent "Node".
[0233] Step 14.1.d.3: In the g level of the "Node Tree", obtain the right-side link pointer array Next[g] of the "Node" to be demoted and obtain the right-side adjacent "Node".
[0234] For example:
[0235] Before demoting a "Node", it is necessary to find the left and right adjacent "Nodes", so steps 14.1.d.2 and 14.1.d.3 are used to find the left and right adjacent "Nodes" that meet the requirements.
[0236] Step 14.1.d.4: Set the right link pointer array Next[g] of the left adjacent "Node" obtained in step 14.1.d.2 to the right adjacent "Node" obtained in step 14.1.d.3; further set the left link pointer array Prev[g] of the right adjacent "Node" obtained in step 14.1.d.3 to the left adjacent "Node" obtained in step 14.1.d.2;
[0237] For example:
[0238] In the process of demoting a "Node", it is necessary to implement the linking and interconnection of the left and right adjacent "Nodes" of the demoted "Node" at level g.
[0239] Step 14.1.d.5: Set both Prev[g] and Next[g] of the "Node" to be downgraded to empty;
[0240] For example:
[0241] Setting the Prev and Next values of the g-th level of a demoted "Node" to empty means that the "Node" no longer has a g-th level.
[0242] Step 14.1.d.6: Downgrade the "Node" to be upgraded to level g-1. The downgrade process is now complete.
[0243] Furthermore, the dynamic adjustment of the "MetaD node set" in step 14.1, which involves "dynamically adjusting the structure" of the Gskiplist subsystem, is performed on a given "Node". If the number of "MetaD nodes" in the "MetaD node set" pointed to by the MetaDID of the "Node" is greater than or equal to the "MetaD node" splitting threshold S in the "Gskiplist parameter file", then the given "Node" becomes the "Node" to be split, which can be split into two new "Nodes" with non-overlapping key value ranges. Based on the key value ranges of the two new "Nodes", the original "MetaD node set" is further split into two new "MetaD node sets", with each new "Node" pointing to a new "MetaD node set". Furthermore, the new "Node" with the key value range on the left inherits the link relationship of the original given "Node" in the "Node tree". The steps for splitting the "MetaD node set" are as follows:
[0244] Step 14.1.s.1: Obtain the key value range [KeyLeft, KeyRight] of the "Node" to be split, and obtain the "MetaD node set" pointed to by the MetaDID in the "Node" to be split;
[0245] Step 14.1.s.2: Sort the “MetaD node set” in step 14.1.s.1 by the key of the “MetaD node”, and select the median value KeyM of the key.
[0246] For example:
[0247] Generally, the median value KeyM of the key is chosen as the splitting key point. Alternatively, the median value or a custom value can be chosen as the splitting key point. The goal is to make the splitting of the "MetaD node set" as uniform as possible, and to make the number of "MetaD nodes" in the two new "MetaD node sets" as similar as possible.
[0248] Step 14.1.s.3: Based on KeyM in step 14.1.s.2, divide the "Node" to be split into two new "Nodes" with key value ranges of [KeyLeft, KeyM) and [KeyM, KeyRight), respectively. The new "Node" on the left inherits the link relationship of the original "Node" in the "Node tree", and the new "Node" on the right is a completely new "Node".
[0249] Step 14.1.s.4: Traverse the "MetaD Node Set" pointed to by the new "Node" on the left in step 14.1.s.3, and migrate all "MetaD Nodes" whose key is not in the key value range [KeyLeft, KeyM) of the new "Node" on the left to the "MetaD Node Set" pointed to by the new "Node" on the right in step 14.1.s.3;
[0250] For example:
[0251] In fact, both the new "Node" on the left and the new "Node" on the right can inherit the link relationships of the original "Node" in the "Node tree".
[0252] Step 14.1.s.5: Complete the splitting of the "MetaD node set". The splitting process is now complete.
[0253] Furthermore, the dynamic adjustment of the "MetaD node set" in step 14.1 of the "dynamic structural adjustment" of the Gskiplist subsystem is for two adjacent "Node nodes" in the level 0 "Node tree". If the number of "MetaD nodes" in the "MetaD node set" pointed to by the MetaDID of these two "Node nodes" is less than the "MetaD node" merging threshold M in the "Gskiplist parameter file", then the two "Node nodes" are merged into a new "Node node", and the corresponding two "MetaD node sets" are also merged together. Furthermore, the link relationship between these two "Node nodes" in the "Node tree" is inherited by the new "Node node". The specific method is to first obtain the highest level of each of the two "Node nodes" in the "Node tree", and then select the "Node node" with the highest level as the new "Node node". If the levels are the same, either "Node node" is selected as the new "Node node", and the link relationship of the other original "Node node" in the "Node tree" is further merged into the new "Node node".
[0254] For example:
[0255] The merging of the "MetaD node set" primarily reduces the memory usage of the Gskiplist subsystem and speeds up retrieval by reducing the number of "Nodes".
[0256] Furthermore, the specific steps of the "Persistent Storage Decision" in the "Persistent Storage Decision and Implementation" of the Gskiplist subsystem described in step 14.2 are as follows:
[0257] Step 14.2.1: First, obtain the actual memory space occupied by the Gskiplist subsystem;
[0258] Step 14.2.2: Then calculate the ratio r of the actual memory space occupied by the Gskiplist subsystem to the "maximum memory space occupied G" in the "Gskiplist parameter file";
[0259] Step 14.2.3: Determine whether the ratio r in step 14.2.2 is greater than or equal to the "persistent storage decision threshold P" in the "Gskiplist parameter file". If it is greater than or equal to, proceed with "persistent storage implementation"; otherwise, do not proceed with "persistent storage implementation".
[0260] For example:
[0261] The "Persistent Storage Decision Threshold P" in the "Gskiplist Parameter File" is generally set to around 0.7. If the ratio r in step 14.2.2 is greater than or equal to 0.7, it means that the specific "persistent storage implementation" can be carried out to save the changed data in memory to the database.
[0262] Step 14.2.4, “Persistent storage decision” processing ends.
[0263] Furthermore, in step 14.2, the "Persistent Storage Implementation" of the Gskiplist subsystem's "Persistent Storage Decision and Implementation" involves using a database as the persistent storage carrier. The database can be a key-value database or a relational database. The database storage format can be key-value (key, value), where key is the key and value is the value corresponding to the key. In the "Node" field, key is NodeID, and value is the "Recursive Length Prefix Encoding" (RLP encoding) of other fields. The value is the result of RLP encoding of the set of all non-NodeID fields in the "Node". In the "MetaD" field, key is MetaDID, and value is the result of RLP encoding of the set of all non-MetaDID fields in the "MetaD" field.
[0264] For example:
[0265] When implementing RLP encoding, the object first needs to be mapped into either a byte array or a list. Taking Go language encoding of "Node" as an example, it will be mapped to the list [NodeID, KeyLeft, KeyRight, MetaDID, SCount, Prev, Next].
[0266] For a "Node", the value calculation process is as follows:
[0267] node:=Node{KeyLeft:"0.01",KeyRight:"0.03",MetaDID:"1234",SCount:"5",Prev:"101",Next:"151"}
[0268] value := bytes.Buffer{}
[0269] rpl.Encode(&value,&node)
[0270] For the "MetaD node", the value calculation process is as follows:
[0271] node:=MetaD{Key:"0.012",NodeID:19,TransactionID:"0x22e11567c38d6e7ea45c0761c13b 08a6cbabdb98461ff05cfdfeb5e1611579b7",BlockID:"9888333",Prev:"1003",Next:"1202"}
[0272] value := bytes.Buffer{}
[0273] rpl.Encode(&value,&node)
[0274] In view of the description and exemplary embodiments of the invention disclosed herein, other embodiments of the invention will be apparent to those skilled in the art. These descriptions and embodiments are considered as examples only, and any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of protection of the invention.
Claims
1. A high-efficiency blockchain query system based on Gskiplist, characterized in that, The system comprises three parts: a blockchain subsystem, a grouped skip list subsystem, and a database. The blockchain subsystem contains all transactions and blocks. Each transaction includes at least a TransactionID, transaction time, transaction participants, transaction header hash, and transaction content information. Each block contains transactions within a given time period. Blocks are linked together in a chain based on their creation time. Each block includes at least a BlockID, block number, block start and end times, the ID of the previous block, the Merkle root of the contained transaction set, the block header hash, and information about the contained transaction set. All transaction and block data in the blockchain subsystem are stored in the database. The grouped skip list subsystem... The system, abbreviated as Gskiplist subsystem, contains a multi-level Node tree composed of Node nodes and multiple MetaD node sets composed of MetaD nodes. Each Node node contains at least the Node ID, KeyLeft (left key), KeyRight (right key), MetaDID (first MetaD node ID), SCount (number of searches), Prev (prev pointer array), and Next (next node pointer array) information. Each Node node also contains [KeyLeft, ...] (the set of nodes whose left key is greater than or equal to the left key and less than the right key). The set of MetaD nodes corresponding to the KeyRight range. Each Node is also linked to other Nodes through pointer arrays Prev and Next, forming multiple levels of chain relationships. Each MetaD node contains at least the MetaD ID, the Key, the Node ID of the corresponding Node, the Transaction ID in the corresponding blockchain, the Block ID in the corresponding blockchain, the MetaDID of the previous MetaD node (Prev), and the MetaDID of the next MetaD node (Next). The Gskiplist subsystem also includes a Gskiplist parameter file to define the multi-level Node tree. The Gskiplist parameter file includes at least the Gskiplist ID (GSL_ID), the Gskiplist search field set (GSL_SearchFields), the maximum level of the Node tree (N), the initial preset level of the Node tree (Q), the initial preset rule type of the Node tree (R), the Node upgrade threshold (U), the Node downgrade threshold (D), the MetaD node fission threshold (S), the MetaD node merging threshold (M), the maximum memory space occupied (G), and the persistent storage decision threshold (P). All Node and MetaD node data in the Gskiplist subsystem are stored in the database.The database is used to store transaction and block data for the blockchain subsystem, and also to store Node and MetaD node data for the Gskiplist subsystem.
2. A dynamic construction method for a blockchain-based high-efficiency query system based on Gskiplist as described in claim 1, characterized in that, The method includes the following specific steps: Step 1: Start the system and set the Gskiplist parameter file to be read; Step 2: Read the Gskiplist parameter file to obtain the Gskiplist ID (GSL_ID), the Gskiplist search field set (GSL_SearchFields), the maximum level of the Node tree (N), the initial preset level of the Node tree (Q), the initial preset rule type of the Node tree (R), the Node upgrade threshold (U), the Node downgrade threshold (D), the MetaD node fission threshold (S), the MetaD node merging threshold (M), the maximum memory space occupied (G), and the persistent storage decision threshold (P). Step 3: Read the blockchain data and build the initial preset grouped skip list subsystem. The specific steps are as follows: Step 3.1: Based on the GSL_SearchFields set of search fields in step 2, read the blockchain data to construct a level 0 Node tree and multiple MetaD node sets; Step 3.2: Based on the initial preset level Q of the Node tree in Step 2, construct the initial preset Node trees from level 1 to level Q on the basis of the level 0 Node tree; Step 4: The system enables blockchain query services to be provided to external parties; Step 5: Monitor the generation of new blocks in the blockchain subsystem and simultaneously monitor and receive query requests from the user. Step 6: Determine the type of the listening response. If it is a new block generated by the blockchain subsystem, proceed to step 17; if it is a query request from the user, proceed to step 7. Step 7: Parse the user's query request; Step 8: Convert the query request in step 7 into the key value range [Vsb, Vse) to be queried; Step 9: Search step by step from the highest level of the Gskiplist subsystem down to level 0, finding all Nodes corresponding to the key value range [Vsb, Vse) given in step 8. The highest level of the Gskiplist subsystem starts at K, but it will change dynamically as the structure of the Gskiplist subsystem changes. The [KeyLeft, KeyRight) of each retrieved Node must have a non-empty intersection with the key value range [Vsb, Vse) given in step 8. Step 10: Among all the Node nodes found in step 9, further search for all MetaD nodes corresponding to the key value range [Vsb, Vse) given in step 8. That is, the key of the retrieved MetaD node must be within the key value range [Vsb, Vse) given in step 8. Step 11: Based on the transaction ID (TransactionID) and block ID (BlockID) information of all MetaD nodes retrieved in step 10, further search for relevant blockchain data in the blockchain. Step 12: Based on all the blockchain data found in step 11, further calculate the final query result of the user's query request; Step 13: For all Node nodes found in step 9, increment the retrieval count (SCount) of each Node node by 1; Step 14: Dynamically adjust the structure of the Gskiplist subsystem using synchronous or asynchronous methods. The specific steps are as follows: Step 14.1: Dynamically adjust the structure of the Gskiplist subsystem using synchronous or asynchronous methods, that is, dynamically adjust the upgrading or downgrading of Node nodes and dynamically adjust the splitting or merging of the MetaD node set; Step 14.2: Make a decision and implement persistent storage for the Gskiplist subsystem using either synchronous or asynchronous methods, and save all in-memory data of the Gskiplist subsystem to the database; Step 15: Return the final query result calculated in step 12 to the user. Step 16: Determine whether the system should continue running. If the system should continue running, proceed to step 5 to continue execution. If the system stops running, proceed to step 18; Step 17: Read the new block of the blockchain and update the content of the Gskiplist subsystem, that is, update multiple MetaD node sets and Node node trees according to the transaction information in the new block; then jump to step 5 to continue execution. Step 18: Implement persistent storage for the Gskiplist subsystem using a synchronous method, saving all in-memory data of the Gskiplist subsystem to the database; Step 19: System execution complete.
3. The method according to claim 2, characterized in that, Step 3.1 describes constructing the level 0 Node tree and multiple MetaD node sets for the Gskiplist subsystem. This involves sequentially reading blockchain blocks and transaction data, and building the level 0 Node tree and multiple MetaD node sets based on the Gskiplist search field set (GSL_SearchFields) from step 2. The following method can be used: Step 3.1.1: Read the block data from block 0 to the largest ID block of the blockchain in sequence. Each block read is abbreviated as Bi, where 0 <= i < the largest ID block. Perform the following processing on each block Bi; Step 3.1.2: Traverse each transaction in block Bi, denoted as Tj, where 0 <= j < the maximum number of transactions in block Bi. Perform the following processing on each transaction Tj; Step 3.1.3: Based on the Gskiplist search field set (GSL_SearchFields) from step 2, calculate the key and value (key, value) for transaction Tj, and denote the result as (Kj, Vj); Step 3.1.4: Create a MetaD node MDj for transaction Tj. The key value of MetaD node MDj is Kj, the TransactionID of MetaD node MDj is the transaction ID of Tj, and the BlockID of MetaD node MDj is the block ID of Tj. Step 3.1.5: In the Gskiplist subsystem, sequentially search the level 0 Node tree to find a Node Nm whose key value range includes Kj. The key value range of this Node Nm is denoted as [x, y), where x <= y; when x equals y, the key value range of Nm only represents x; initially, the level 0 Node tree has only one Node, whose key value range is [-∞, +∞). Step 3.1.6: Insert the MetaD node MDj information from step 3.1.4 into the MetaD node set of Node node Nm from step 3.1.5; Step 3.1.7: Calculate the number of MetaD nodes owned by Node Nm in step 3.1.
5. If the number of MetaD nodes owned by Node Nm reaches the MetaD node fission threshold S read in step 2, a value t can be calculated so that Node Nm can be fissioned into two new Nodes. The key value ranges of the two new Nodes are [x, t) and [t, y), respectively. The principle of choosing the value t is to make the two Nodes have approximately the same number of MetaD nodes. Step 3.1.8: Determine whether all transactions in block Bi have been traversed. If all transactions in block Bi have been traversed, proceed to step 3.1.9 to continue execution; otherwise, proceed to step 3.1.2 to process the next transaction. Step 3.1.9: Determine whether all blocks in the blockchain have been traversed. If all blocks in the blockchain have been traversed, proceed to step 3.1.10 to continue execution; otherwise, proceed to step 3.1.1 to process the next block. Step 3.1.10: Processing ends.
4. The method according to claim 2, characterized in that, Step 3.2 describes constructing the initial preset Node tree for levels 1 to K of the Gskiplist subsystem. This is done by upgrading some Nodes in the level 0 Node tree according to the initial preset level Q and the initial preset rule type R of the Node tree in the Gskiplist parameter file, thereby progressively building the Node tree for levels 1 to K. The initial preset rule type R of the Node tree can include the following rule types: binary rule, ternary rule, N-partary rule, and custom rule. The binary rule refers to upgrading one Node from every two consecutive Nodes in the same level Node tree; the ternary rule refers to upgrading one Node from every three consecutive Nodes in the same level Node tree; the N-partary rule refers to upgrading one Node from every N consecutive Nodes in the same level Node tree; and the custom rule refers to selecting several Nodes in the same level Node tree according to a rule defined by the designer for upgrading.
5. The method according to claim 2, characterized in that, Step 14.1 describes the dynamic structural adjustment of the Gskiplist subsystem, specifically the dynamic adjustment and upgrading of Node nodes. This is based on whether the retrieval count (SCount) of each Node node meets the Node node upgrade threshold U condition in the Gskiplist parameter file. If the upgrade condition is met, the Node node is promoted one level in the Node node tree. The steps for promoting a Node node are as follows: Step 14.1.u.1: Obtain the highest level of the Node to be upgraded, denoted as g, and upgrade the Node to be upgraded to level g+1; Step 14.1.u.2: In the g level of the Node tree, obtain the left-side link pointer array Prev[g] of the Node to be upgraded and obtain the left-side adjacent Node; if the left-side adjacent Node does not have a g+1 level, continue to obtain the left-side adjacent Node of the left-side adjacent Node until a left-side adjacent Node with a g+1 level is obtained. Step 14.1.u.3: In the g level of the Node tree, obtain the right-side link pointer array Next[g] of the Node to be upgraded and obtain the right-side adjacent Node; if the right-side adjacent Node does not have a g+1 level, continue to obtain the right-side adjacent Node of the right-side adjacent Node until a right-side adjacent Node with a g+1 level is obtained. Step 14.1.u.4: Set the right link pointer array Next[g+1] of the left adjacent Node obtained in step 14.1.u.2 as the Node to be upgraded; further set the left link pointer array Prev[g+1] of the right adjacent Node obtained in step 14.1.u.3 as the Node to be upgraded. Step 14.1.u.5: Further set the left link pointer array Prev[g+1] of the Node to be upgraded to the left adjacent Node obtained in step 14.1.u.2, and set the right link pointer array Next[g+1] of the Node to be upgraded to the right adjacent Node obtained in step 14.1.u.3; Step 14.1.u.6: Complete the upgrade of the Node to be upgraded to level g+1. The upgrade process is now complete.
6. The method according to claim 2, characterized in that, Step 14.1 describes the dynamic structural adjustment of the Gskiplist subsystem, specifically the dynamic adjustment of Node demotion. This is based on whether the retrieval count (SCount) of each Node meets the Node demotion threshold D in the Gskiplist parameter file. If the demotion condition is met, the Node is demoted one level in the Node tree. The steps for demoting a Node by one level are as follows: Step 14.1.d.1: Obtain the highest level of the Node to be downgraded, denoted as g, and intend to downgrade the Node to level g-1; Step 14.1.d.2: In the g level of the Node tree, obtain the left-side link pointer array Prev[g] of the Node to be demoted and obtain the left-side adjacent Node; Step 14.1.d.3: In the g level of the Node tree, obtain the right-side link pointer array Next[g] of the Node to be demoted and obtain the right-side adjacent Node; Step 14.1.d.4: Set the right link pointer array Next[g] of the left adjacent Node obtained in step 14.1.d.2 to the right adjacent Node obtained in step 14.1.d.3; further set the left link pointer array Prev[g] of the right adjacent Node obtained in step 14.1.d.3 to the left adjacent Node obtained in step 14.1.d.
2. Step 14.1.d.5: Set both Prev[g] and Next[g] of the node to be downgraded to null; Step 14.1.d.6: Complete the downgrade of the Node to be upgraded to level g-1, and the downgrade process is complete.
7. The method according to claim 2, characterized in that, Step 14.1, which describes the dynamic structural adjustment of the Gskiplist subsystem, specifically the splitting of the MetaD node set, involves, for a given Node, if the number of MetaD nodes in the MetaD node set pointed to by the MetaDID of that Node is greater than or equal to the MetaD node splitting threshold S in the Gskiplist parameter file, then the given Node becomes the Node to be split. It can be split into two new Nodes with non-overlapping key value ranges. Based on the key value ranges of the two new Nodes, the original MetaD node set is further split into two new MetaD node sets, with each new Node pointing to a new MetaD node set. Furthermore, the new Node with the key value range on the left inherits the link relationships of the original given Node in the Node tree. The steps for splitting the MetaD node set are as follows: Step 14.1.s.1: Obtain the key value range [KeyLeft, KeyRight] of the Node to be split, and obtain the set of MetaD nodes pointed to by the MetaDID in the Node to be split; Step 14.1.s.2: Sort the set of MetaD nodes in step 14.1.s.1 by the key of the MetaD nodes, and select the median value KeyM of the key. Step 14.1.s.3: According to KeyM in step 14.1.s.2, the Node to be split is divided into two new Nodes with key value ranges of [KeyLeft, KeyM) and [KeyM, KeyRight), respectively. The new Node on the left inherits the link relationship of the original Node in the Node tree, and the new Node on the right is a completely new Node. Step 14.1.s.4: Traverse the set of MetaD nodes pointed to by the new Node on the left in step 14.1.s.3, and migrate all MetaD nodes whose key is not in the key value range [KeyLeft, KeyM) of the new Node on the left to the set of MetaD nodes pointed to by the new Node on the right in step 14.1.s.
3. Step 14.1.s.5: Complete the splitting of the MetaD node set; the splitting process is now complete.
8. The method according to claim 2, characterized in that, The dynamic adjustment of the MetaD node set merging described in step 14.1 for the dynamic structural adjustment of the Gskiplist subsystem is as follows: For two adjacent Node nodes in the 0-level Node node tree, if the number of MetaD nodes in the MetaD node set pointed to by the MetaDID of these two Node nodes is less than the MetaD node merging threshold M in the Gskiplist parameter file, then these two Node nodes are merged into a new Node node, and the corresponding two MetaD node sets are also merged together. Furthermore, the link relationship between these two Node nodes in the Node node tree is inherited by the new Node node. The specific method is to first obtain the highest level of each of the two Node nodes in the Node node tree, and then select the Node node with the highest level as the new Node node. If the levels are the same, then any Node node is selected as the new Node node, and further, the link relationship of the other original Node node in the Node node tree is merged into the new Node node.
9. The method according to claim 2, characterized in that, The specific steps for making and implementing persistent storage decisions for the Gskiplist subsystem as described in step 14.2 are as follows: Step 14.2.1: First, obtain the actual memory space occupied by the Gskiplist subsystem; Step 14.2.2: Then calculate the ratio r of the actual memory space occupied by the Gskiplist subsystem to the maximum memory space occupied value G in the Gskiplist parameter file; Step 14.2.3: Determine whether the ratio r in step 14.2.2 is greater than or equal to the persistent storage decision threshold P in the Gskiplist parameter file. If it is greater than or equal to the threshold P, then persistent storage is implemented; otherwise, persistent storage is not implemented. Step 14.2.4: Persistent storage decision processing ends.
10. The method according to claim 2, characterized in that, In step 14.2, the persistent storage implementation of the Gskiplist subsystem involves making persistent storage decisions. This invention uses a database as the persistent storage carrier. The database can be a key-value database or a relational database. The database storage format can be key-value (key, value), where key is the key and value is the value corresponding to the key. In a Node, the key is NodeID, and the value is the recursive length prefix code (RLP code) of other fields. The value is the result of RLP encoding of the set of all non-NodeID fields in the Node. In a MetaD node, the key is MetaDID, and the value is the result of RLP encoding of the set of all non-MetaDID fields in the MetaD node.
Citation Information
Patent Citations
Data storage and query method and device, electronic equipment and readable storage medium
CN111666289A
System or method for implementing forgotten rights on metadata-driven blockchains using secret sharing and consensus of reads
CN114365133A