Blockchain architecture implementation method supporting on-chain and off-chain collaborative expansion

Abstract

The present application relates to a kind of support on-chain off-chain collaborative extension of storage-computing separation blockchain architecture implementation method, it is related to blockchain field.The present application constructs on-chain computing node layer, on-chain storage node layer and off-chain database layer, each layer is realized data communication and collaboration by cross-layer interaction module;Design node role division mechanism based on resource quantification score and double threshold stable determination, according to node hardware resources and runtime load, dynamically divide node into on-chain computing node, on-chain storage node and on-chain buffer candidate node;Establish library chain collaborative data storage and verification mechanism;For on-chain computing node layer and on-chain storage node layer, establish independent dynamic expansion and contraction mechanism, on-chain computing node layer uses micro-batch pipeline execution and account address hash fragment to improve parallel capability, on-chain storage node layer uses differentiating fragmentation strategy for different types of data, and data routing is managed by global configuration table.Through active lease mechanism, cross-layer data consistency is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of blockchain distributed system construction technology, and in particular to a method for implementing a storage-computation separation blockchain architecture that supports on-chain and off-chain collaborative expansion. Background Technology

[0002] Blockchain technology, with its distributed ledger, cryptographic primitives, and consensus mechanisms, provides fundamental support for trusted collaboration in multi-party, distrustful environments. However, traditional blockchains employ a monolithic architecture that couples storage and computation, requiring all full nodes to simultaneously undertake the dual responsibilities of smart contract execution and complete ledger storage. This results in severely insufficient system scalability: when computational pressure surges, it cannot scale up computing power individually; when data volume increases, it cannot scale up storage individually, and can only expand by adding full nodes as a whole, leading to significant resource waste.

[0003] While existing storage-compute separation solutions decouple computation and storage to some extent, they still have significant drawbacks: First, node role assignments often rely on static configurations, making it impossible to dynamically adjust based on runtime load and adapt to fluctuations in business traffic. Second, they lack native support for large-scale unstructured data; the high storage costs of blockchain itself prevent it from handling high-definition multimedia, large documents, and other similar data. Third, frequent cross-network interactions between computing and storage nodes after storage-compute separation make it difficult to guarantee state data consistency, leading to dirty reads and data drift issues. Fourth, existing solutions do not fully leverage the high-performance storage advantages of traditional databases, resulting in insufficient depth of database-chain collaboration and making it difficult to meet the needs of trusted sharing of massive amounts of data in data element circulation scenarios.

[0004] Therefore, there is an urgent need to design a library-chain collaborative storage-computation separation architecture that can dynamically divide node roles, support trusted storage of large files, ensure state consistency, and achieve independent elastic scaling of computing and storage. Summary of the Invention

[0005] To address the aforementioned technical problems, or at least partially address them, this invention provides a method for implementing a storage-computation separation blockchain architecture that supports on-chain and off-chain collaborative expansion.

[0006] This invention provides a method for implementing a storage-computation separation blockchain architecture that supports on-chain and off-chain collaborative scaling, including: A three-layer blockchain architecture with storage and computation separation that supports on-chain and off-chain collaborative expansion is constructed, including: an on-chain computing node layer, an on-chain storage node layer, and an off-chain database layer from top to bottom. Each layer achieves data communication and collaboration through cross-layer interaction modules. To support the collaborative expansion of on-chain and off-chain storage and computation separation blockchain architecture, a node role division mechanism based on resource quantification scoring and dual threshold stability determination is designed. According to the node hardware resources and runtime load, the nodes are dynamically divided into on-chain computing nodes, on-chain storage nodes, and on-chain buffer candidate nodes. To support the collaborative expansion of on-chain and off-chain storage and computational separation blockchain architecture, a collaborative data storage and verification mechanism between the library and the blockchain is established. The off-chain database implements large file block storage, while the on-chain only stores semantic quintuples. On-chain computing nodes verify the integrity of off-chain files by performing full hash recalculation through semantic quintuples. To support the collaborative scaling of on-chain and off-chain storage-separated blockchain architecture, an independent dynamic scaling mechanism is established between the on-chain computing node layer and the on-chain storage node layer. The on-chain computing node layer adopts micro-batch pipeline execution and account address hash sharding to improve parallelism. The on-chain storage node layer adopts differentiated sharding strategies for different types of data and manages data routing through a global configuration table.

[0007] Furthermore, the on-chain computing node layer consists of multiple on-chain computing node groups, each containing a set number of on-chain computing nodes responsible for transaction verification, smart contract execution, consensus building, and block packaging; the on-chain computing nodes locally deploy data caching modules and file caching modules to accelerate access to hot data; The on-chain storage node layer consists of multiple on-chain storage node groups. Each on-chain storage node group adopts a redundant architecture of one primary and two backups, which is responsible for the persistent storage of ledger data, state data, contract code and off-chain file tuples. The on-chain storage nodes build independent MPT subtrees for different types of data for state verification. The off-chain database layer is built on MongoDB and uses the GridFS mechanism to implement block storage and retrieval of large files. It establishes a trusted mapping relationship with the on-chain system through semantic quintuples.

[0008] Furthermore, the node role division mechanism based on resource quantification scoring and dual-threshold stability determination specifically includes: Establish a resource quantification scoring model to calculate the computing power score C of on-chain nodes by comprehensively considering four indicators: CPU core count, clock speed, memory capacity, and network bandwidth; and calculate the storage capacity score S of on-chain nodes by comprehensively considering four indicators: disk capacity, IOPS, sequential throughput, and network bandwidth. The ratio of computing power to storage capacity is R = C / S; Set a dual-rating ratio threshold, denoted as the first rating ratio threshold. Second score ratio threshold Second rating ratio threshold Less than the first score ratio threshold ; When the score ratio is greater than or equal to the first score ratio threshold When the on-chain node is divided into on-chain computing nodes; when the scoring ratio is less than or equal to the second scoring ratio threshold... At that time, the on-chain nodes are divided into on-chain storage nodes, and the remaining nodes are stored in the node pool as on-chain buffer candidate nodes to wait for deployment.

[0009] Furthermore, the library chain collaborative data storage and verification mechanism specifically includes: When uploading large files, the on-chain compute nodes stream the file to the file system of the off-chain database layer. The file system automatically performs file block storage, obtains the unique physical index of the returned file: the file route, and generates a global hash digest of the file through an encryption algorithm. On-chain computing nodes construct a semantic quintuple consisting of file route, file global hash digest, file name, file type, and semantic tag. At the same time, they generate a globally unique identifier for the file. Using the globally unique identifier as the key and the semantic quintuple as the value, the semantic quintuple is written to the on-chain storage node through a transaction to complete persistence. When accessing a file stored in the off-chain database layer, the on-chain compute node first obtains the semantic quintuple of the file identifier UID in the corresponding access request from the on-chain storage node, and then pulls the corresponding complete file from the database of the off-chain database layer according to the file route in the semantic quintuple. On-chain compute nodes recalculate the file hash locally based on the complete file and compare it with the file's global hash digest in the semantic quintuple stored by the on-chain storage nodes. Only after successful verification can the file data be used.

[0010] Furthermore, the dynamic scaling mechanism of the on-chain computing node layer specifically includes: The computing node group is configured with one master node, at least two backup nodes and one verification node. It adopts a micro-batch pipeline to realize a parallel processing mode of execution and consensus on the same side, and combines a sharding mechanism based on account address hash to improve transaction concurrency processing capability. When the system detects that the CPU utilization of the computing node group continuously exceeds the set first utilization threshold or the transaction queue delay exceeds the set duration, it triggers expansion, allocates idle on-chain buffer candidate nodes from the node pool to form a new on-chain computing node group, and transfers part of the account scope under the responsibility of the overloaded group to the new group. When the CPU utilization of multiple on-chain compute node groups is consistently below the second utilization threshold, a scaling-down mechanism is triggered, merging the account range of the low-load group into other on-chain compute node groups, and reclaiming the idle on-chain nodes after account allocation to the node pool. Furthermore, during the scaling process of the on-chain computing node layer, the new on-chain computing node group downloads the latest global sharding configuration table and block header chain, and prefetches hot data based on the cache heat information of the overload group; after the overload group completes the current micro-batch, it updates the global sharding configuration table, and subsequent transactions are directly routed to the new on-chain computing node group for execution; During the scaling down of the on-chain computing node layer, the global sharding configuration table is updated, and the account scope under the responsibility of the low-load group is divided into other target on-chain computing node groups. The low-load group stops receiving new transactions, and after processing the backlog of transactions, it forwards the execution results to the on-chain computing node group. The on-chain computing nodes of the low-load group clear their memory, disconnect from the network, and return to the node pool to wait.

[0011] Furthermore, the micro-batch pipeline implements a parallel processing mode of execution-on-the-fly consensus, with on-chain compute node groups and on-chain storage node groups working together to achieve a complete transaction processing flow, including: pre-maintaining a global sharding configuration table and a global storage configuration table, recording the transaction execution scope of the compute node group and the data storage scope of the storage node group, respectively; the client forwards the transaction to the master node of the corresponding on-chain compute node group according to the global sharding configuration table; the master node of the on-chain compute node group verifies the transaction signature and format, checks whether the local cache has the required data for execution, and if not, requests the data from the on-chain storage node; the chain... The master node of the on-chain compute node group executes transactions to generate a read-write set, which is then broadcast to the backup nodes within the group for verification. After successful verification, the master node of the on-chain compute node group packages the transactions into micro-batches, which are then asynchronously replayed by the verification nodes, which extract a set proportion of the transactions. Once the master node of the on-chain compute node group has collected enough signatures, it sends the semi-finished block and read-write set to the corresponding on-chain storage node group. The on-chain storage nodes verify the read-write set, update the state data, and generate a new state root, which is then inserted into the block header. The on-chain storage nodes persist the complete block, and the on-chain compute nodes broadcast the new block header to the entire network.

[0012] Furthermore, the dynamic scaling mechanism of the on-chain storage node layer specifically includes: The on-chain storage node group adopts a redundant architecture of one primary and two backups; A differentiated sharding strategy is adopted for ledger data, state data, contract code, and off-chain indexes; When the disk utilization rate of the on-chain storage node group exceeds the first disk utilization rate threshold or the state read continues to time out, expansion is triggered. The on-chain storage node layer expansion obtains on-chain buffer candidate nodes from the node pool to form a new on-chain storage node group, and completes the data migration and routing switch to the new on-chain node group through snapshot plus incremental synchronization. When the load of an on-chain storage node group remains low for an extended period, a scaling-down mechanism is triggered. This mechanism merges the node's data into adjacent on-chain storage node groups whose combined disk usage is below a preset threshold, and then reclaims the idle nodes to the node pool.

[0013] Furthermore, during the scaling up of the on-chain storage node layer, the overloaded original on-chain storage node group creates a snapshot of its current state and sends it to the new on-chain storage node group. During the snapshot transmission, the original on-chain storage node group will synchronously forward new writes for the partitioned data to the new on-chain storage node group. The new on-chain storage node group imports the snapshot and replays the incremental logs. After the state is consistent with that of the original on-chain storage node group, it updates the global storage configuration table and takes over the read and write requests for some data. During the scaling down of the on-chain storage node layer, a failure notification is sent to the compute nodes that hold the original on-chain storage node group data.

[0014] Furthermore, an active lease mechanism is introduced to ensure the consistency of state data. This active lease mechanism specifically includes: Each on-chain storage node group maintains an active lease table locally, recording the on-chain computing node group ID, node IP, lease token, data version number, lease activation and expiration time information of the on-chain computing node group holding state data cache; When an on-chain compute node requests specific state data from an on-chain storage node, the on-chain storage node returns the target data and grants a lease with the corresponding validity period, and records the relevant information in its local active lease table. When the state data is updated due to a new block confirmation, the on-chain storage node immediately updates the data version number and proactively pushes a data expiration notification to all on-chain compute nodes holding valid leases. Upon receiving the notification, the on-chain compute node immediately marks the corresponding local cache as expired, thus ensuring strong consistency of state data in a distributed environment with extremely low network overhead.

[0015] The technical solutions provided in the embodiments of the present invention have the following advantages compared with the prior art: This invention achieves complete decoupling of computing and storage resources and independent elastic scaling, which can dynamically adjust resource allocation according to business load, effectively improving the problems of rigid resource configuration and uneven utilization in traditional storage-computing coupled architecture.

[0016] This application constructs a deeply integrated library-chain collaboration system, which utilizes MongoDB's high-performance storage capabilities to handle large files, and ensures data immutability through on-chain hash anchoring, thus solving the problem that blockchain cannot support unstructured data.

[0017] This application employs an active lease mechanism to ensure strong consistency of state data with extremely low network overhead, avoiding common data inconsistency problems in storage-compute separation architectures.

[0018] The micro-batch pipeline and sharding mechanism of the on-chain computing node layer in this application significantly improves the parallel processing capability of transactions, and the system throughput is increased by more than 90% compared with the traditional architecture. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic diagram of a storage-computation separation blockchain architecture that supports on-chain and off-chain collaborative expansion, provided as an embodiment of the present invention; Figure 2 This is a flowchart of the complete transaction processing method provided in the embodiments of the present invention; Figure 3 This is a flowchart of the on-chain computing node group expansion provided in an embodiment of the present invention; Figure 4 This is a flowchart of the on-chain computing node group scaling down provided in an embodiment of the present invention; Figure 5 This is a flowchart of the on-chain storage node group expansion provided in an embodiment of the present invention; Figure 6 This is a flowchart of the scaling down of the on-chain storage node group provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0024] Example 1 This invention proposes a method for implementing a storage-computation separation blockchain architecture that supports on-chain and off-chain collaborative scaling. It decouples computing and storage resources through a resource-aware dynamic node grouping mechanism, introduces MongoDB as an off-chain collaborative storage layer to handle large files, and ensures cross-layer data consistency through an active lease mechanism. Simultaneously, it achieves independent elastic scaling of the computing and storage layers. The technical solution of this invention's storage-computation separation blockchain architecture implementation method includes: like Figure 1 As shown, a three-layered blockchain architecture supporting on-chain and off-chain collaborative expansion is constructed, comprising an on-chain computing node layer, an on-chain storage node layer, and an off-chain database layer, arranged from top to bottom. Each layer achieves data communication and collaboration through a cross-layer interaction module. A global configuration module maintains a global sharding configuration table and a global storage configuration table, while a data management module performs data reading and writing and generates state roots.

[0025] In practice, the on-chain computing node layer consists of multiple on-chain computing node groups, each containing four on-chain computing nodes, responsible for transaction verification, smart contract execution, consensus building, and block packaging. On-chain computing nodes locally deploy data and file caches to accelerate access to frequently accessed data.

[0026] The on-chain storage node layer consists of multiple on-chain storage node groups. Each on-chain storage node group adopts a redundant architecture of one primary and two backup, namely one primary on-chain storage node and two backup on-chain storage nodes, which are responsible for the persistent storage of ledger data, state data, contract code and off-chain file tuples. The on-chain storage nodes build independent MPT subtrees for different types of data for state verification.

[0027] Off-chain database layer: Built on MongoDB, using the GridFS file system to achieve block storage and efficient retrieval of large files, and establishing a trusted mapping relationship with the on-chain system (on-chain compute node layer and on-chain storage node layer) through semantic quintuples.

[0028] To support the collaborative expansion of on-chain and off-chain storage and computation separation blockchain architecture, a node role division mechanism based on resource quantification scoring and dual threshold stability determination is designed. According to the node hardware resources and runtime load, the nodes are dynamically divided into on-chain computing nodes, on-chain storage nodes, and on-chain buffer candidate nodes.

[0029] The node role division mechanism based on resource quantification scoring and dual-threshold stability determination specifically includes: Establish a resource quantification scoring model to calculate the computing power score C of on-chain nodes by comprehensively considering four indicators: CPU core count, clock speed, memory capacity, and network bandwidth; and calculate the storage capacity score S of on-chain nodes by comprehensively considering four indicators: disk capacity, IOPS, sequential throughput, and network bandwidth. The ratio of computing power to storage capacity, R=C / S, reflects the computing performance that can be provided per unit of storage performance.

[0030] Set a dual-rating ratio threshold, denoted as the first rating ratio threshold. Second score ratio threshold Example The value is 1.2. The value is 0.8; When the score ratio is greater than or equal to the first score ratio threshold When the on-chain node is divided into on-chain computing nodes; when the scoring ratio is less than or equal to the second scoring ratio threshold... At that time, the on-chain nodes are divided into on-chain storage nodes, and the remaining nodes are stored in the node pool as on-chain buffer candidate nodes to wait for deployment.

[0031] The node role allocation mechanism adopts a dynamic load correction mechanism, which updates the node capability score by combining real-time CPU utilization and disk utilization, and avoids frequent role switching caused by short-term load fluctuations through a stable window mechanism, thus ensuring the stability of system operation.

[0032] To support the collaborative expansion of on-chain and off-chain storage and computational separation blockchain architecture, a collaborative data storage and verification mechanism between the library and the chain is established. The off-chain database uses MongoDB GridFS to implement large file block storage, while the on-chain database only stores semantic quintuples containing information such as file routing and hash digest. The computing nodes verify the integrity of off-chain files through full hash recalculation.

[0033] The library chain collaborative data storage and verification mechanism specifically includes: When uploading large files, the on-chain compute nodes stream the files to the MongoDB database's GridFS file system in the off-chain database layer. The GridFS file system automatically performs file block storage and obtains the unique physical index Route_ID of the returned file, i.e., the file route. The on-chain compute nodes also generate a global hash digest of the file, File_Hash, using the SHA-256 encryption algorithm.

[0034] On-chain computing nodes construct a semantic quintuple consisting of file route Route_ID, file global hash digest File_Hash, file name File_Name, file type File_Type, and semantic tags Semantic_Tags, and generate a globally unique identifier (UID) for the file. Using the file's globally unique identifier as the key and the semantic quintuple as the value, the semantic quintuple is written to the on-chain storage node through a transaction to complete persistence. When accessing a file stored in the off-chain database layer, the on-chain compute node first obtains the semantic quintuple of the globally unique identifier (UID) in the corresponding access request from the on-chain storage node, and then pulls the corresponding complete file from the off-chain database layer MongoDB according to the file route (Route_ID). On-chain compute nodes recalculate the file hash locally based on the complete file and strictly compare it with the file's global hash digest in the semantic quintuple stored by the on-chain storage nodes. Only after successful verification can the file data be used.

[0035] To support the collaborative scaling of on-chain and off-chain storage-separated blockchain architecture, an independent dynamic scaling mechanism is established between the on-chain computing node layer and the on-chain storage node layer. The on-chain computing node layer adopts micro-batch pipeline execution and account address hash sharding to improve parallelism. The on-chain storage node layer adopts differentiated sharding strategies for different types of data and manages data routing through a global configuration table.

[0036] The independent dynamic scaling mechanism of the on-chain computing node layer and the on-chain storage node layer is detailed.

[0037] Specifically, the dynamic scaling mechanism of the on-chain computing node layer includes: The compute node group is configured with one master node, at least two standby nodes and one verification node. The example on-chain compute node group is configured with one master node, two standby nodes and one verification node. It adopts a micro-batch pipeline to realize the parallel processing mode of execution and consensus on the same side, and combines a sharding mechanism based on account address hash to improve the transaction concurrency processing capability. When the system detects that the CPU utilization of the compute node group continuously exceeds the set first utilization threshold, or the transaction queue latency exceeds the set duration, it triggers expansion. In this example, the first utilization threshold is 80%, and the transaction queue latency is set to 20 seconds. The expansion process allocates idle on-chain buffer candidate nodes from the node pool to form a new on-chain compute node group, and transfers a portion of the account scope handled by the overloaded group to the new on-chain compute node group, thereby achieving transaction migration.

[0038] When the CPU utilization of multiple on-chain compute node groups remains below the second utilization threshold, a scaling-down is triggered. In this example, the second utilization threshold is 30%. The scaling-down process merges the account range of the low-load group into other on-chain compute node groups and reclaims the idle on-chain nodes after the account allocation to the node pool.

[0039] The micro-batch pipeline achieves a parallel processing mode of execution-on-demand consensus through collaborative execution by on-chain computing node groups and on-chain storage node groups, such as... Figure 2As shown, the complete transaction processing flow includes: S21: The client forwards the transaction to the master node of the corresponding on-chain compute node group according to the global sharding configuration table. S22: The master node of the on-chain compute node group verifies the transaction signature and format, checks if the local cache has the required data, and if not, requests data from the on-chain storage node or off-chain database. S23: The master node of the on-chain compute node group executes the transaction, generates the transaction result and read / write set, and broadcasts it to the backup nodes within the on-chain compute node group for verification and generation of a semi-finished block. S24: After successful verification, the master node of the on-chain compute node group packages the transaction into micro-batches, and the verification nodes extract a set proportion of transactions for asynchronous replay; in this example, the proportion is 10%. S25: After collecting enough signatures, the master node of the on-chain compute node group sends the semi-finished block and read / write set to the corresponding on-chain storage node group. S26: On-chain storage nodes verify the read / write set, update the state data, and the data management module generates a new state root, which is then filled into the block header. S27: On-chain storage nodes persist the complete block, and on-chain compute nodes broadcast the new block header to the entire network.

[0040] like Figure 3 As shown, the expansion of on-chain compute node groups is triggered by the system monitoring and scheduling module and applied in the elastic scaling scenario of on-chain compute node layers. This includes: S31: The system monitors transaction throughput, on-chain compute node CPU utilization, and transaction latency in real time. When the CPU utilization of an on-chain compute node group continuously exceeds a preset first utilization threshold or the transaction queue latency exceeds a set duration, the expansion process is triggered. S32: Four idle on-chain buffer candidate nodes are allocated from the node pool to form a new on-chain compute node group. The new on-chain compute node group downloads the latest global sharding configuration table and block header chain, achieving global sharding configuration and state synchronization. S33: Half of the account scope managed by the overload group is allocated to the new on-chain compute node group. The new on-chain compute node group prefetches hot data based on the cache heat information of the overload group. S34: After the overload group completes the current micro-batch, it updates the global sharding configuration table, and subsequent transactions are directly routed to the new on-chain compute node group for execution.

[0041] like Figure 4As shown, the scaling down of on-chain compute node groups is triggered by the system monitoring and scheduling module and is applied in scenarios of elastic scaling up and down at the compute layer. This includes: S41: Real-time monitoring of on-chain compute node groups. When the CPU utilization of two or more on-chain compute node groups remains below a preset threshold, the scaling down process is triggered. S42: Identifying low-load on-chain compute node groups, allocating the account scope handled by the low-load group to other target on-chain compute node groups, migrating the transaction traffic (i.e., account sharding) of the low-load group to the target on-chain compute node groups, and updating the global sharding configuration table. S43: The low-load group stops receiving new transactions, processes the backlog of transactions, and forwards the execution results to the target on-chain compute node groups. S44: The on-chain compute nodes in the low-load group clear their memory, disconnect from the network, return to the node pool to wait, complete the scaling down, and update the global resource status.

[0042] The dynamic scaling mechanism of the on-chain storage node layer specifically includes: The storage nodes are grouped into threes and employ a redundant architecture with one primary and two backup nodes. A differentiated sharding strategy is adopted for ledger data, state data, contract code, and off-chain indexes; Expansion is triggered when the disk utilization of an on-chain storage node group exceeds the first disk utilization threshold or when state reads continuously time out. For example, expansion is triggered when the disk utilization of a storage group exceeds 85% or when state reads continuously time out.

[0043] The on-chain storage node layer expansion obtains on-chain buffer candidate nodes from the node pool to form a new on-chain storage node group, and completes the data migration and routing switch to the new on-chain node group through snapshot plus incremental synchronization.

[0044] When the load of an on-chain storage node group remains low for an extended period, a scaling down mechanism is triggered. Data is then merged into adjacent on-chain storage node groups, and idle nodes are reclaimed and returned to the node pool.

[0045] like Figure 5 As shown, the expansion of the on-chain storage node group is triggered by the system monitoring and scheduling module and is applied to the elastic scaling scenarios of the on-chain storage node layer, including: S51: Monitor the status of on-chain storage node groups in real time. When the disk utilization of an on-chain storage node group exceeds the preset first disk utilization threshold or the state read continues to time out, trigger the expansion process. S52: Select three on-chain buffer candidate nodes from the node pool to establish a new on-chain storage node group. The overloaded original on-chain storage node group creates a current state snapshot and sends it to the new on-chain storage node group. S53: During snapshot transmission, the original on-chain storage node group synchronously forwards new write requests for partitioned data to the new on-chain storage node group. S54: The new on-chain storage node group imports the snapshot and replays the incremental logs until the state alignment verification is consistent with the original on-chain storage node group. It then updates the global storage configuration table, switches read / write routes to the new on-chain storage node group, and completes the expansion.

[0046] like Figure 6 As shown, the scaling down of the on-chain storage node group is triggered by the system monitoring and scheduling module and is applied to elastic scaling up and down scenarios at the on-chain storage node layer, including: S61: Monitor the status of on-chain storage node groups in real time. When the disk utilization of an on-chain storage node group is lower than a preset threshold and the request volume remains low, trigger the scaling-down process. S62: Filter low-load on-chain storage node groups and determine target storage node groups that can be merged. Specifically, select adjacent groups whose combined disk utilization is lower than a preset threshold as target on-chain storage node groups. S63: Migrate the data from the low-load group to the target on-chain storage node group and update the global storage configuration table. S64: Send failure notifications to the compute nodes holding data from the original on-chain storage node group. S65: The low-load original on-chain storage node group deletes the migrated data. If it no longer stores any data, it returns to the node pool to wait.

[0047] An active lease mechanism is introduced to ensure the consistency of state data. On-chain storage nodes maintain an active lease table and proactively push failure notifications to on-chain computing nodes holding valid leases when state data is updated.

[0048] The active lease mechanism specifically includes: each on-chain storage node group maintains an active lease table locally, recording the on-chain computing node group ID, node IP, lease token, data version number, lease activation and expiration time information of the on-chain computing node group holding the state data cache; When an on-chain compute node requests specific state data from an on-chain storage node, the on-chain storage node returns the target data and grants a lease with the corresponding validity period, and records the relevant information in its local active lease table. When the state data is updated due to a new block confirmation, the on-chain storage node immediately updates the data version number and proactively pushes a data expiration notification to all on-chain compute nodes holding valid leases. Upon receiving the notification, the on-chain compute node immediately marks the corresponding local cache as expired, thus ensuring strong consistency of state data in a distributed environment with extremely low network overhead.

[0049] In the embodiments provided by this invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, structures, or units, and may be electrical, mechanical, or other forms.

[0050] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0051] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0052] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for implementing a storage-computation separation blockchain architecture that supports on-chain and off-chain collaborative scaling, characterized in that, include: A three-layer blockchain architecture with storage and computation separation that supports on-chain and off-chain collaborative expansion is constructed, including: an on-chain computing node layer, an on-chain storage node layer, and an off-chain database layer from top to bottom. Each layer achieves data communication and collaboration through cross-layer interaction modules. To support the collaborative expansion of on-chain and off-chain storage and computation separation blockchain architecture, a node role division mechanism based on resource quantification scoring and dual threshold stability determination is designed. According to the node hardware resources and runtime load, the nodes are dynamically divided into on-chain computing nodes, on-chain storage nodes, and on-chain buffer candidate nodes. To support the collaborative expansion of on-chain and off-chain storage and computational separation blockchain architecture, a collaborative data storage and verification mechanism between the library and the blockchain is established. The off-chain database implements large file block storage, while the on-chain only stores semantic quintuples. On-chain computing nodes verify the integrity of off-chain files by performing full hash recalculation through semantic quintuples. To support the collaborative scaling of on-chain and off-chain storage-separated blockchain architecture, an independent dynamic scaling mechanism is established between the on-chain computing node layer and the on-chain storage node layer. The on-chain computing node layer adopts micro-batch pipeline execution and account address hash sharding to improve parallelism. The on-chain storage node layer adopts differentiated sharding strategies for different types of data and manages data routing through a global configuration table.

2. The method for implementing a storage-computation separation blockchain architecture supporting on-chain and off-chain collaborative expansion according to claim 1, characterized in that, The on-chain computing node layer consists of multiple on-chain computing node groups, each containing a set number of on-chain computing nodes, responsible for transaction verification, smart contract execution, consensus achievement, and block packaging; the on-chain computing nodes locally deploy data caching modules and file caching modules to accelerate access to hot data; The on-chain storage node layer consists of multiple on-chain storage node groups. Each on-chain storage node group adopts a redundant architecture of one primary and two backups, which is responsible for the persistent storage of ledger data, state data, contract code and off-chain file tuples. The on-chain storage nodes build independent MPT subtrees for different types of data for state verification. The off-chain database layer is built on MongoDB and uses the GridFS mechanism to implement block storage and retrieval of large files. It establishes a trusted mapping relationship with the on-chain system through semantic quintuples.

3. The method for implementing a storage-computation separation blockchain architecture supporting on-chain and off-chain collaborative expansion according to claim 1, characterized in that, The node role division mechanism based on resource quantification scoring and dual-threshold stability determination specifically includes: Establish a resource quantification scoring model to calculate the computing power score C of on-chain nodes by comprehensively considering four indicators: CPU core count, clock speed, memory capacity, and network bandwidth; and calculate the storage capacity score S of on-chain nodes by comprehensively considering four indicators: disk capacity, IOPS, sequential throughput, and network bandwidth. The ratio of computing power to storage capacity is R = C / S; Set a dual-rating ratio threshold, denoted as the first rating ratio threshold. Second score ratio threshold Second rating ratio threshold Less than the first score ratio threshold ; When the score ratio is greater than or equal to the first score ratio threshold When the on-chain node is divided into on-chain computing nodes; when the scoring ratio is less than or equal to the second scoring ratio threshold... At that time, the on-chain nodes are divided into on-chain storage nodes, and the remaining nodes are stored in the node pool as on-chain buffer candidate nodes to wait for deployment.

4. The method for implementing a storage-computation separation blockchain architecture supporting on-chain and off-chain collaborative expansion according to claim 1, characterized in that, The library chain collaborative data storage and verification mechanism specifically includes: When uploading large files, the on-chain compute nodes stream the file to the file system of the off-chain database layer. The file system automatically performs file block storage, obtains the unique physical index of the returned file: the file route, and generates a global hash digest of the file through an encryption algorithm. On-chain computing nodes construct a semantic quintuple consisting of file route, file global hash digest, file name, file type, and semantic tag. At the same time, they generate a globally unique identifier for the file. Using the globally unique identifier as the key and the semantic quintuple as the value, the semantic quintuple is written to the on-chain storage node through a transaction to complete persistence. When accessing a file stored in the off-chain database layer, the on-chain compute node first obtains the semantic quintuple of the file identifier UID in the corresponding access request from the on-chain storage node, and then pulls the corresponding complete file from the database of the off-chain database layer according to the file route in the semantic quintuple. On-chain compute nodes recalculate the file hash locally based on the complete file and compare it with the file's global hash digest in the semantic quintuple stored by the on-chain storage nodes. Only after successful verification can the file data be used.

5. The method for implementing a storage-computation separation blockchain architecture supporting on-chain and off-chain collaborative expansion according to claim 1, characterized in that, The dynamic scaling mechanism of the on-chain computing node layer specifically includes: The computing node group is configured with one master node, at least two backup nodes and one verification node. It adopts a micro-batch pipeline to realize a parallel processing mode of execution and consensus on the same side, and combines a sharding mechanism based on account address hash to improve transaction concurrency processing capability. When the system detects that the CPU utilization of the computing node group continuously exceeds the set first utilization threshold or the transaction queue delay exceeds the set duration, it triggers expansion, allocates idle on-chain buffer candidate nodes from the node pool to form a new on-chain computing node group, and transfers part of the account scope under the responsibility of the overloaded group to the new group. When the CPU utilization of multiple on-chain compute node groups remains below the second utilization threshold, a scaling-down mechanism is triggered, merging the account range of the low-load groups into other on-chain compute node groups, and reclaiming the idle on-chain nodes after account allocation to the node pool.

6. The method for implementing a storage-computation separation blockchain architecture supporting on-chain and off-chain collaborative expansion according to claim 5, characterized in that, During the scaling process of the on-chain computing node layer, the new on-chain computing node group downloads the latest global sharding configuration table and block header chain, and prefetches hot data based on the cache heat information of the overload group; after the overload group completes the current micro-batch, it updates the global sharding configuration table, and subsequent transactions are directly routed to the new on-chain computing node group for execution; During the scaling down of the on-chain computing node layer, the global sharding configuration table is updated, and the account scope under the responsibility of the low-load group is divided into other target on-chain computing node groups. The low-load group stops receiving new transactions, and after processing the backlog of transactions, it forwards the execution results to the on-chain computing node group. The on-chain computing nodes of the low-load group clear their memory, disconnect from the network, and return to the node pool to wait.

7. The method for implementing a storage-computation separation blockchain architecture supporting on-chain and off-chain collaborative expansion according to claim 1, characterized in that, The micro-batch pipeline implements a parallel processing mode of execution-while-consensus, with on-chain compute node groups and on-chain storage node groups working together to achieve a complete transaction processing flow, including: pre-maintaining a global sharding configuration table and a global storage configuration table, recording the transaction execution scope of the compute node group and the data storage scope of the storage node group, respectively; the client forwards the transaction to the master node of the corresponding on-chain compute node group according to the global sharding configuration table; the master node of the on-chain compute node group verifies the transaction signature and format, checks whether the local cache has the required data for execution, and if not, requests the data from the on-chain storage node; on-chain computation... The master node of the node group executes transactions to generate a read / write set, which is then broadcast to the backup nodes within the on-chain computing node group for verification. After successful verification, the master node of the on-chain computing node group packages the transactions into micro-batches, and the verification nodes extract a set proportion of these transactions for asynchronous replay. Once the master node of the on-chain computing node group has collected enough signatures, it sends the semi-finished block and read / write set to the corresponding on-chain storage node group. The on-chain storage nodes verify the read / write set, update the state data, and generate a new state root, which is then filled into the block header. The on-chain storage nodes persist the complete block, and the on-chain computing nodes broadcast the new block header to the entire network.

8. The method for implementing a storage-computation separation blockchain architecture supporting on-chain and off-chain collaborative expansion according to claim 1, characterized in that, The dynamic scaling mechanism of the on-chain storage node layer specifically includes: The on-chain storage node group adopts a redundant architecture of one primary and two backups; A differentiated sharding strategy is adopted for ledger data, state data, contract code, and off-chain indexes; When the disk utilization rate of the on-chain storage node group exceeds the first disk utilization rate threshold or the state read continues to time out, expansion is triggered. The on-chain storage node layer expansion obtains on-chain buffer candidate nodes from the node pool to form a new on-chain storage node group, and completes the data migration and routing switch to the new on-chain node group through snapshot plus incremental synchronization. When the load of an on-chain storage node group remains low for an extended period, a scaling-down mechanism is triggered. This mechanism merges the node's data into adjacent on-chain storage node groups whose combined disk usage is below a preset threshold, and then reclaims the idle nodes to the node pool.

9. The method for implementing a storage-computation separation blockchain architecture supporting on-chain and off-chain collaborative expansion according to claim 8, characterized in that, During the scaling up of the on-chain storage node layer, the overloaded original on-chain storage node group creates a snapshot of its current state and sends it to the new on-chain storage node group. During the snapshot transmission, the original on-chain storage node group will synchronously forward new writes of partitioned data to the new on-chain storage node group. The new on-chain storage node group imports the snapshot and replays the incremental logs. After the state is consistent with that of the original on-chain storage node group, it updates the global storage configuration table. During the scaling down of the on-chain storage node layer, a failure notification is sent to the compute nodes that hold the original on-chain storage node group data.

10. The method for implementing a storage-computation separation blockchain architecture supporting on-chain and off-chain collaborative expansion according to claim 1, characterized in that, An active lease mechanism is introduced to ensure the consistency of state data. This active lease mechanism specifically includes: Each on-chain storage node group maintains an active lease table locally, recording the on-chain computing node group ID, node IP, lease token, data version number, lease activation and expiration time information of the on-chain computing node group holding state data cache; When an on-chain compute node requests specific state data from an on-chain storage node, the on-chain storage node returns the target data and grants a lease with the corresponding validity period, and records the relevant information in its local active lease table. When the state data is updated due to a new block confirmation, the on-chain storage node immediately updates the data version number and proactively pushes a data expiration notification to all on-chain compute nodes holding valid leases. Upon receiving the notification, the on-chain compute node immediately marks the corresponding local cache as expired, thus ensuring strong consistency of state data in a distributed environment with low network overhead.

Images