A blockchain-based trusted data distributed storage acceleration system
Patent Information
- Application Number
- CN202611331605.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-31
- Publication Date
- 2026-11-03
Smart Images
Figure CN123030682A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data storage technology, specifically to a blockchain-based trusted distributed data storage acceleration system. Background Technology
[0002] With its immutable and fully traceable characteristics, blockchain technology has become a crucial technological support for the storage of compliant business data such as contracts and operation logs. The combination of distributed storage and blockchain provides a technological foundation for the on-chain storage of massive amounts of compliant data. Currently, the industry often combines techniques such as hierarchical storage with hot and cold data separation, Merkle tree integrity verification, lightweight node scheduling, and data compression and decompression to try to optimize the storage and access performance of blockchain storage systems, aiming to balance the strong trustworthiness requirements of compliant data storage with system operating efficiency. The integrated application of these technologies has become an important development direction in the field of blockchain data storage.
[0003] In practical applications, existing technologies have not yet achieved deep adaptation between various technical modules and the specific characteristics of compliant data storage. General-purpose Merkle trees are not optimized for the structured and hierarchical management characteristics of compliant data, resulting in high computational overhead for data verification. Layered storage strategies lack effective coordination with the blockchain's trusted verification logic; cross-layer migration of hot and cold data can easily affect the stability of the hash structure, making it difficult to simultaneously meet the requirements of storage optimization and the continuity of trusted data storage. The synchronization and scheduling of lightweight nodes often adopt general solutions, leading to high resource overhead from full ledger synchronization and requiring improvement in node response efficiency. Data compression and hash verification are mostly executed serially without customized block processing based on data aggregation characteristics. Overall processing efficiency is insufficient to meet the needs of batch storage and auditing of massive amounts of compliant data, resulting in continued problems of storage capacity expansion and limited access and verification efficiency even after massive amounts of compliant data are uploaded to the blockchain. To address these issues, we propose a blockchain-based trusted data distributed storage acceleration system. Summary of the Invention
[0004] To address the aforementioned technical issues, a blockchain-based trusted data distributed storage acceleration system is provided. This technical solution resolves the problems of existing technologies where modules are not deeply adapted to the characteristics of compliant data storage, the overhead of general Merkle tree verification is high, the coordination between hierarchical storage and trusted verification is insufficient, the resource overhead of lightweight node synchronization is large, and the serial execution efficiency of data compression and hash verification is low. These issues lead to storage expansion and limited access and verification efficiency after massive amounts of compliant data are uploaded to the blockchain.
[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A blockchain-based trusted distributed data storage acceleration system includes: Blockchain evidence storage module, hierarchical storage module, lightweight node scheduling module, parallel compression module; The blockchain evidence storage module is used to build a trusted evidence storage ledger for compliant data based on an improved Merkle tree structure, and generate a unique root hash covering all compliant data. The hierarchical storage module is coupled with the blockchain evidence storage module to separate and store hot and cold data according to the importance level and access frequency of compliant data, and to perform cross-level data migration. The lightweight node scheduling module is coupled with the hierarchical storage module and the blockchain evidence storage module, and is used to perform dynamic scheduling of lightweight nodes and on-demand data synchronization based on the node load status. The parallel compression module is coupled with the hierarchical storage module and the blockchain evidence storage module, and is used to perform block-based parallel compression and decompression processing on compliant data. The structure of the improved Merkle tree corresponds to the hot and cold data hierarchy of the hierarchical storage module. When data is migrated across levels, the root hash of the improved Merkle tree remains unchanged.
[0006] Preferably, the process of constructing a trusted ledger for compliant data based on an improved Merkle tree structure, and generating a unique root hash covering all compliant data, specifically involves: Obtain all compliant data to be uploaded to the blockchain, and extract and mark four basic attributes for each piece of compliant data: compliance level, data subject, access frequency, and retention period. Based on the basic attributes of the compliant data that have been marked, an improved Merkle tree hierarchical structure corresponding to the cold and hot data hierarchy of the hierarchical storage module is constructed to complete the leaf node aggregation processing of the compliant data. Following the hierarchical structure of the improved Merkle tree, the aggregate hash calculation of each level of nodes is completed from bottom to top, generating the intermediate node hash of the corresponding level. Based on the aggregated hash of all hierarchical intermediate nodes, a unique global root hash covering all compliant data is generated, thus completing the construction of a trusted ledger for compliant data storage. When compliant data is migrated across levels, only the storage level to which the compliant data belongs is adjusted, without changing the hash value of each node in the improved Merkle tree, thus keeping the global root hash constant.
[0007] Preferably, the hierarchical structure of the improved Merkle tree is constructed as follows: A leaf node layer is constructed to aggregate compliant data with the same compliance level and the same data subject into the same leaf node, and a single compliant data is not treated as an independent leaf node. An intermediate aggregation layer is constructed, which is divided into three levels of intermediate nodes according to the access frequency and retention period of compliant data. The three levels of intermediate nodes correspond one-to-one with the hot data, warm data and cold data levels of the tiered storage module. Each intermediate node stores the aggregate hash of all leaf nodes in the corresponding tier. A root node layer is constructed, and a unique global root hash is generated based on the aggregated hash of the three-level intermediate nodes. The global root hash covers the integrity information of all compliant data. Establish hierarchical binding rules to permanently bind the three-level intermediate nodes to the corresponding hot and cold data levels. The migration of compliant data across levels does not change the hash value of the corresponding node or its level.
[0008] Preferably, the hash calculation and data integrity verification of the improved Merkle tree specifically involves: For fixed fields in compliant data, pre-hash caching is performed, and fixed fields of the same type of compliant data are hashed only once. Parallel hash calculations are performed on multiple aggregate nodes at the same level of the improved Merkle tree; For integrity verification of a single piece of compliant data, obtain the hash value of the leaf node to which the compliant data belongs, and complete the hash comparison verification along the path of the corresponding intermediate node; For integrity verification of batch compliant data in the same dimension, obtain the hash value of the corresponding aggregate leaf node of the batch of compliant data, and complete the hash comparison verification along a single path; For compliant data after cross-level migration, integrity verification is performed using the node hash path corresponding to the data before migration.
[0009] Preferably, the separation and storage of hot and cold data according to the importance level and access frequency of compliant data specifically refers to: Establish data classification rules based on both compliance importance level and access frequency, dividing compliance data into three levels: hot data, warm data, and cold data. These three data levels correspond one-to-one with the three intermediate nodes of the improved Merkle tree. Establish the criteria for dividing each data level, and clarify the compliance level requirements, access frequency thresholds, and statutory retention period requirements for each level; Match the corresponding storage medium to each data level to complete the initial storage deployment of compliant data at each level; Based on the dynamic changes in the attributes of compliance data, perform cross-level migration of compliance data and synchronously update the storage location of compliance data; Configure the corresponding read / write access interface according to the storage tier to which the compliant data belongs.
[0010] Preferably, the cross-level migration of execution data specifically includes: Based on three preset trigger logics, it is determined whether the compliant data meets the conditions for cross-level migration. The three trigger logics include access frequency trigger, compliance cycle trigger, and capacity threshold trigger. For compliant data that meets the conditions for downgrade migration, perform migration operations from the hot data layer to the warm data layer, or from the warm data layer to the cold data layer. For compliant data that meets the upgrade and migration conditions, perform migration operations from the cold data layer to the warm data layer, or from the warm data layer to the hot data layer. For compliant data during the migration process, storage deployment is completed by matching the storage media corresponding to the target tier; After completing the cross-level migration of compliant data, the integrity of the migrated compliant data is verified by using the hash value of the corresponding node in the improved Merkle tree.
[0011] Preferably, the dynamic scheduling of lightweight nodes and on-demand data synchronization based on node load status specifically refers to: We construct a load assessment index system for lightweight nodes, setting four core assessment dimensions: the cache carrying capacity of the data at the corresponding node level, hash verification computing power, online stability, and task response latency. The system collects load data from all lightweight nodes in the scheduling pool in real time and calculates the overall health and task handling capacity of each node based on the load evaluation index system. Receive access and verification requests for compliant data, match available lightweight nodes at the corresponding level according to the storage level and operation type of the request, and assign corresponding processing tasks. Real-time monitoring of the load status and task execution of lightweight nodes; dynamic migration and load balancing adjustment of execution tasks when the node load exceeds a preset threshold. Based on the overall health changes of lightweight nodes, perform full lifecycle management of execution nodes.
[0012] Preferably, the execution rules for the dynamic scheduling and on-demand data synchronization of the lightweight nodes are as follows: For lightweight nodes applying to join the scheduling pool, an admission verification is performed, which includes verifying the legitimacy of the node's identity and the resource carrying capacity of the corresponding level. For lightweight nodes that pass the admission verification, perform initial data synchronization. The synchronization content includes the global root hash of the improved Merkle tree and the intermediate node hash of the node's corresponding access level. After the synchronization is completed, the node is added to the scheduling pool to undertake processing tasks. For lightweight nodes that have been added to the scheduling pool, incremental data synchronization is performed. Incremental hash data and corresponding hot data are synchronized only when the hash of the intermediate node of the improved Merkle tree at the corresponding level is updated. For lightweight nodes that meet the exit conditions, a smooth exit process is executed. First, the cached data and processing tasks of the corresponding level of the node are migrated to other available nodes. After the handover is completed, the node is removed from the scheduling pool. For lightweight nodes that exit abnormally, disaster recovery measures are implemented, migrating the unfinished processing tasks and corresponding cached data of the node to a standby available node.
[0013] Preferably, the process of performing block-based parallel compression and decompression on compliant data specifically involves: Based on the aggregation boundaries of the leaf nodes of the improved Merkle tree, independent compression blocks are divided, and aggregated compliant data within the same leaf node are divided into the same independent compression block; Each independent compressed block is assigned a separate computing thread to perform parallel compression processing, generating the corresponding compressed package and compressed block hash value; The hash value of the compressed block is included in the hash calculation range of the corresponding leaf node, thus completing the binding of the compressed data with the trusted evidence storage ledger; When receiving an access request for compliant data, decompression is performed only on the compressed block containing the target compliant data; During the decompression process, the integrity of compliant data is simultaneously verified using the hash value of the compressed block.
[0014] Preferably, the algorithm adaptation and cooperative execution rules for the block-based parallel compression and decompression processing are as follows: For compliant data in the hot data layer, the LZ4 algorithm, modified for parallel processing, is used to perform compression and decompression. For compliant data in both the warm and cold data layers, a parallelized DEFLATE algorithm is used, along with a dedicated static dictionary for compliant data, to perform compression and decompression. The dedicated static dictionary for compliant data is pre-built based on fixed fields of the compliant data. During the compression process, the data stream of the compressed block is synchronously input into the compression thread and the hash calculation thread, and the compression process and hash calculation are executed synchronously. During the decompression process, the decompression data stream is synchronously input into the decompression thread and the hash verification thread, and the decompression process and integrity verification are executed synchronously. After compression or decompression, the integrity of all compliant data is verified by using the hash value of the corresponding node in the improved Merkle tree.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The trusted data distributed storage acceleration system proposed in this invention achieves a dual improvement in storage efficiency and data trustworthiness through an innovative hierarchical aggregation Merkle tree structure adapted to the characteristics of compliant data. The system effectively solves the problem of storage capacity expansion after massive amounts of compliant business data are uploaded to the blockchain, while significantly improving data access and verification efficiency. Through a layered storage and dynamic migration mechanism for hot and cold data, the system can intelligently allocate storage resources according to the importance level and access frequency of the data, ensuring efficient access to hot data and economical storage of cold data. The dynamic scheduling of lightweight nodes and the on-demand data synchronization strategy greatly reduce node overhead and improve the overall response speed of the system. The introduction of parallel compression and decompression algorithms further optimizes storage space utilization and data processing efficiency, providing strong technical support for the trusted storage of compliant data. Attached Figure Description
[0016] Figure 1 This is a system module diagram of the present invention. Detailed Implementation
[0017] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.
[0018] Reference Figure 1 As shown, a blockchain-based trusted distributed data storage acceleration system includes: Blockchain evidence storage module, hierarchical storage module, lightweight node scheduling module, parallel compression module; The blockchain evidence storage module is used to build a trusted evidence storage ledger for compliant data based on an improved Merkle tree structure, and generate a unique root hash covering all compliant data. The process of constructing a trusted ledger for compliant data based on an improved Merkle tree structure, and generating a unique root hash covering all compliant data, specifically involves: Obtain all compliant data to be uploaded to the blockchain, extract and mark four basic attributes for each piece of compliant data: compliance level, data subject, access frequency, and retention period. The compliance level is divided into core level and general level according to the legal requirements of the business. The data subject includes the unique identifier such as the enterprise and system to which the data belongs. The access frequency is counted according to the number of reads and writes per unit time. The retention period is defined according to the legal compliance archiving requirements. Based on the basic attributes of the compliant data that have been marked, an improved Merkle tree hierarchical structure corresponding to the cold and hot data hierarchy of the hierarchical storage module is constructed to complete the leaf node aggregation processing of compliant data. The aggregation processing strictly follows the aggregation rules of the same compliance level and the same data subject, and does not perform data aggregation across subjects or levels. Following the hierarchical structure of the improved Merkle tree, the aggregate hash calculation for each level of nodes is performed from bottom to top, generating the hash of the intermediate node at the corresponding level. The aggregate hash calculation uses the formula... ,in The aggregate hash value of the current node. to This represents the hash values of all child nodes under the current node. For byte concatenation operations, the aggregate hash calculation of nodes at the same level adopts a multi-threaded parallel execution method; Based on the aggregated hash of all hierarchical intermediate nodes, a unique global root hash covering all compliant data is generated through the above aggregated hash calculation formula, thus completing the construction of a trusted evidence storage ledger for compliant data. The ledger permanently records the binding relationship between the hash values of each level node and the data attributes. When compliant data is migrated across levels, only the storage level to which the compliant data belongs is adjusted. The hash values of each node in the improved Merkle tree are not changed, and the global root hash remains constant. The migration operation only modifies the data storage location identifier and does not affect the hash record information in the ledger.
[0019] The hierarchical structure of the improved Merkle tree is constructed as follows: A leaf node layer is constructed to aggregate compliance data with the same compliance level and the same data subject into the same leaf node. A single compliance data is not treated as an independent leaf node. A single leaf node can aggregate multiple compliance data under the same level of the contract subject, which greatly reduces the total number of leaf nodes. An intermediate aggregation layer is constructed, which is divided into three levels of intermediate nodes according to the access frequency and retention period of compliant data. The three levels of intermediate nodes correspond one-to-one with the hot data, warm data and cold data levels of the tiered storage module. Each intermediate node stores the aggregate hash of all leaf nodes in the corresponding tier. The hash value of the three levels of intermediate nodes is only updated when leaf nodes are added or deleted in the corresponding tier. Data migration across tiers does not trigger changes in the hash value of intermediate nodes. A root node layer is constructed. Based on the aggregated hash of the three-level intermediate nodes, a unique global root hash is generated through the aggregated hash calculation formula. The global root hash covers the integrity information of all compliant data. The global root hash is only updated synchronously when the hash value of the three-level intermediate nodes is updated. After the update, it is permanently recorded in the blockchain ledger. Establish hierarchical binding rules to permanently bind three levels of intermediate nodes to corresponding hot and cold data levels. The migration of compliant data across levels does not change the hash value and level of the corresponding node. The binding rules are written into the blockchain smart contract, which automatically executes and verifies the validity of the binding relationship.
[0020] The hash calculation and data integrity verification of the improved Merkle tree are specifically as follows: Pre-hashing caching is performed on fixed fields in compliant data. Fixed fields of the same type of compliant data are hashed only once. The hash calculation for fixed fields uses the following formula. ,in For the pre-hash value of a fixed field, For fixed field content in compliant data, the pre-hash value is stored in a distributed cache, and subsequent data of the same type directly calls the pre-hash value in the cache. Parallel hash calculations are performed on multiple aggregate nodes at the same level of the improved Merkle tree. The corresponding computing threads are allocated according to the server's computing power configuration. Each thread is responsible for the hash calculation of one or more aggregate nodes, and there is no data dependency between threads. For integrity verification of a single piece of compliant data, the hash value of the leaf node to which the compliant data belongs is obtained, and the hash comparison verification is completed along the path of the corresponding intermediate node. Only the single path from the leaf node to the root node corresponding to the data is traversed, without traversing the entire tree node. For integrity verification of batch compliance data in the same dimension, obtain the hash value of the corresponding aggregate leaf node of the batch compliance data, and complete the hash comparison verification along a single path. Contract data is aggregated by the enterprise and year, and operation logs are aggregated by the system and time period. Batch verification only needs to verify the hash path from the corresponding aggregate leaf node to the root node. For compliant data after cross-level migration, the integrity verification is completed using the node hash path corresponding to the original node before migration. The compliant data after migration still maintains the hash binding relationship with the original leaf nodes and intermediate nodes, and the verification path is completely consistent with that before migration. For contract data verification, single contract verification only obtains the leaf node hash of the compliance level of the entity to which it belongs, and the batch audit of enterprise annual contracts only obtains the aggregated leaf node hash of the corresponding year of the enterprise. The verification path length is shortened by more than 90% compared with the traditional Merkle tree. For operation log verification, daily or monthly batch operation log verification only requires obtaining the aggregated leaf node hash of the corresponding time period. Completing single-path verification is sufficient to confirm the integrity of the entire log, without needing to calculate the hash of each log entry individually.
[0021] The hierarchical storage module is coupled with the blockchain evidence storage module to separate and store hot and cold data according to the importance level and access frequency of compliant data, and to perform cross-level data migration. The specific steps for separating and storing hot and cold data according to the importance level and access frequency of compliant data are as follows: Establish a data classification rule based on both compliance importance level and access frequency. Divide compliance data into three levels: hot data, warm data, and cold data. The three data levels correspond one-to-one with the three intermediate nodes of the improved Merkle tree. The two-dimensional classification rule is solidified through smart contracts and automatically determines the level of compliance data. Establish the classification criteria for each data level, and clarify the compliance level requirements, access frequency thresholds, and statutory retention period requirements for each level. Hot data is core compliant data that is accessed frequently within 7 days; warm data is general compliant data that is accessed moderately within 30 days; and cold data is archived compliant data that is accessed infrequently for more than 180 days. Core compliant data includes original contracts, judicial evidence, and signature data; general compliant data includes operation logs and approval records; and archived compliant data is data that needs to be permanently archived after exceeding the statutory retention period. Each data tier is matched with a corresponding storage medium to complete the initial storage deployment of compliant data at each tier. The hot data tier uses a distributed cache cluster plus high-performance distributed block storage, the warm data tier uses a Ceph / IPFS distributed file system, and the cold data tier uses low-cost archive-type object storage. Each storage medium is selected according to the performance requirements of the tier, taking into account both access efficiency and storage cost. Based on the dynamic changes in the attributes of compliance data, cross-level migration of compliance data is performed, and the storage location of compliance data is updated synchronously. Changes in data attributes are monitored in real time by the system, and the monitoring indicators include changes in access frequency and retention period expiration status. Configure corresponding read and write access interfaces according to the storage tier to which the compliant data belongs. Configure low-latency, high-concurrency read and write interfaces for hot data tiers, balanced read and write interfaces for warm data tiers, and high-capacity, low-frequency access interfaces for cold data tiers. The interface permissions should match the data compliance level.
[0022] The cross-level migration of execution data specifically refers to: According to the three preset trigger logics, it is determined whether the compliant data meets the conditions for cross-level migration. The three trigger logics include access frequency trigger, compliance cycle trigger, and capacity threshold trigger. Access frequency trigger is when the read and write frequency is lower than or higher than the preset threshold within a continuous period. Compliance cycle trigger is when the data expires according to the legal data retention requirements. Capacity threshold trigger is when the capacity of the hot data layer or warm data layer exceeds the set threshold. For compliant data that meets the conditions for downgrade migration, perform migration operations from the hot data layer to the warm data layer or from the warm data layer to the cold data layer. Downgrade migration prioritizes the migration of low-frequency and non-core compliant data to ensure that the storage resource utilization rate of the hot data layer is within a reasonable range. For compliant data that meets the upgrade and migration conditions, perform migration operations from the cold data layer to the warm data layer or from the warm data layer to the hot data layer. Upgrade and migration are for archived data or general compliant data that are suddenly accessed at high frequency. After the migration is completed, the access interface configuration is updated synchronously. For compliant data during the migration process, storage deployment is completed by matching the storage media corresponding to the target level. The migration process adopts the breakpoint resume method to avoid data loss caused by data transmission interruption. The hash binding relationship of the data remains unchanged during the migration process. After completing the cross-level migration of compliant data, the integrity of the migrated compliant data is verified by using the hash value of the corresponding node in the improved Merkle tree. If the verification passes, the data storage level identifier is updated; if the verification fails, the data rollback mechanism is triggered to restore the data to its original storage location.
[0023] The lightweight node scheduling module is coupled with the hierarchical storage module and the blockchain evidence storage module, and is used to perform dynamic scheduling of lightweight nodes and on-demand data synchronization based on the node load status. The specific steps for performing dynamic scheduling and on-demand data synchronization of lightweight nodes based on node load status are as follows: A load assessment metric system for lightweight nodes is constructed, defining four core assessment dimensions: caching capacity of the node's corresponding data layer, hash verification computing power, online stability, and task response latency. The overall health of the node is assessed using a formula. Calculation, where To assess the overall health of the nodes, Score the cache capacity. Score the hash verification computing power. For online stability rating, Score the task response latency. These are the weight coefficients for each dimension, and The weighting coefficients are dynamically configured based on the business scenario; The system collects load data from all lightweight nodes in the scheduling pool in real time. The collection cycle is set according to business needs. Based on the load assessment index system, the system calculates the overall health and task carrying capacity of each node. The task carrying capacity is determined by the overall health and the percentage of remaining resources of the node. Upon receiving access and verification requests for compliant data, the system matches available lightweight nodes at the corresponding level based on the storage level and operation type of the request, and assigns corresponding processing tasks. Access requests are preferentially assigned to nodes with better cache capacity, and verification requests are preferentially assigned to nodes with stronger hash verification computing power. Real-time monitoring of the load status and task execution of lightweight nodes. When the node load exceeds the preset threshold, dynamic migration and load balancing of tasks are performed. Task migration adopts a seamless migration method without interrupting the processing of user requests. Based on the changes in the overall health of lightweight nodes, the entire lifecycle management of the nodes is implemented. Nodes with an overall health above a preset threshold continue to operate normally, nodes within the threshold range have their task allocation reduced, and nodes below the threshold trigger an exit process.
[0024] The specific execution rules for the dynamic scheduling and on-demand data synchronization of the lightweight nodes are as follows: For lightweight nodes applying to join the scheduling pool, an admission verification is performed. The verification includes the legitimacy of the node's identity and the resource carrying capacity of the corresponding level. The legitimacy of the identity is completed through the blockchain node identity authentication mechanism, and the resource carrying capacity verification verifies whether the node's hardware configuration meets the processing requirements of the corresponding level. For lightweight nodes that pass the admission verification, perform initial data synchronization. The synchronization content includes the global root hash of the improved Merkle tree and the intermediate node hash of the node's corresponding access level. After the synchronization is completed, the node is added to the scheduling pool to undertake processing tasks. The initial synchronization does not require synchronizing the full ledger data, but only the hash data of the corresponding level. For lightweight nodes that have been added to the scheduling pool, incremental data synchronization is performed. The incremental hash data and the corresponding hot data are synchronized only when the hash of the intermediate node of the improved Merkle tree at the corresponding level is updated. The hash of the intermediate node corresponding to cold data is constant and does not need to be synchronized repeatedly. Incremental synchronization adopts an event-triggered method, and synchronization is automatically triggered when the hash is updated. For lightweight nodes that meet the exit conditions, a smooth exit process is executed. First, the cached data and processing tasks of the corresponding level of the node are migrated to other available nodes. After the handover is completed, the node is removed from the scheduling pool. The exit conditions are that the overall health of the node is continuously lower than the preset threshold or the node actively initiates an exit application. For lightweight nodes that exit abnormally, disaster recovery is performed, migrating the unfinished processing tasks and corresponding cached data of the node to a standby available node. Abnormal exit includes situations such as node offline and hardware failure. Disaster recovery is automatically triggered by the scheduling module. Lightweight nodes and full nodes adopt an on-demand incremental synchronization strategy. Full nodes only store the full ledger and hash data at each level, while lightweight nodes only synchronize the hash data and hot data at the level corresponding to the tasks they undertake, which greatly reduces the data synchronization overhead between nodes.
[0025] The parallel compression module is coupled with the hierarchical storage module and the blockchain evidence storage module, and is used to perform block-based parallel compression and decompression processing on compliant data. The specific steps for performing block-based parallel compression and decompression of compliant data are as follows: Based on the aggregation boundaries of the leaf nodes of the improved Merkle tree, independent compressed blocks are divided. Aggregated compliant data within the same leaf node are divided into the same independent compressed block. The division of compressed blocks corresponds one-to-one with the leaf nodes, and compressed blocks are not divided across leaf nodes to ensure the binding relationship between compressed blocks and hash nodes. Each independent compressed block is assigned an independent computing thread to perform parallel compression processing, generating a corresponding compressed package and compressed block hash value. The compressed block hash value is calculated using the SHA256 algorithm. The number of threads is dynamically allocated according to the number of server cores to ensure the parallel efficiency of compression processing. The hash value of the compressed block is included in the hash calculation range of the corresponding leaf node, thus completing the binding of compressed data with the trusted evidence ledger. The aggregate hash calculation of the leaf node includes the hash value of the compressed block, and the integrity of the compressed data is directly related to the ledger hash. When receiving access requests for compliant data, decompression is only performed on the compressed block containing the target compliant data, without decompressing the entire data. Single-block decompression significantly reduces the computational overhead of decompression and improves data access efficiency. During the decompression process, the integrity of compliant data is simultaneously verified using the hash value of the compressed block. Decompression and verification are performed in parallel. If the verification fails, the data is rejected and a data anomaly alarm is triggered.
[0026] The algorithm adaptation and cooperative execution rules for the block-based parallel compression and decompression processing are as follows: For compliant data in the hot data layer, the parallelized LZ4 algorithm is used to perform compression and decompression processing. The parallelized LZ4 algorithm optimizes the data block and thread scheduling logic, taking into account both compression speed and decompression efficiency, and adapting to the low latency requirements of high-frequency reading and writing of hot data. For compliance data in the warm and cold data layers, a parallel DEFLATE algorithm is used, along with a dedicated static dictionary for compliance data to perform compression and decompression. The dedicated static dictionary for compliance data is pre-built based on fixed fields of compliance data and includes high-frequency repetitive content such as fixed contract clauses and fixed fields of operation logs. The dictionary content is dynamically updated according to the business data type. During the compression process, the data stream of the compressed block is synchronously input into the compression thread and the hash calculation thread, and the compression and hash calculation are executed synchronously. The data stream adopts a double buffer mechanism, and the two threads read the same data stream at the same time without any order of execution, which improves the efficiency of the entire chain. During the decompression process, the decompressed data stream is synchronously input into the decompression thread and the hash verification thread, and the decompression process and integrity verification are executed synchronously. The verification thread compares the hash value of the decompressed data with the hash value of the compressed block stored in the ledger in real time to ensure that the data has not been tampered with. After compression or decompression, the integrity of all compliant data is verified by using the hash value of the corresponding node in the improved Merkle tree. Once the verification is successful, the compressed data is stored in the corresponding storage medium or the decompressed data is returned to the requester. For structured compliant data, a compression ratio of no less than 3:1 is achieved through block parallel processing and optimization with a dedicated static dictionary. Compared with the traditional serial compression scheme, the computational processing efficiency is improved by more than 30%, and the parallel execution of compression and hashing further shortens the processing time for data to be uploaded to the blockchain.
[0027] The structure of the improved Merkle tree corresponds to the hot and cold data hierarchy of the hierarchical storage module. When data is migrated across levels, the root hash of the improved Merkle tree remains unchanged.
[0028] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.
Claims
1. A blockchain-based trusted distributed data storage acceleration system, characterized in that, include: Blockchain evidence storage module, hierarchical storage module, lightweight node scheduling module, parallel compression module; The blockchain evidence storage module is used to build a trusted evidence storage ledger for compliant data based on an improved Merkle tree structure, and generate a unique root hash covering all compliant data. The hierarchical storage module is coupled with the blockchain evidence storage module to separate and store hot and cold data according to the importance level and access frequency of compliant data, and to perform cross-level data migration. The lightweight node scheduling module is coupled with the hierarchical storage module and the blockchain evidence storage module, and is used to perform dynamic scheduling of lightweight nodes and on-demand data synchronization based on the node load status. The parallel compression module is coupled with the hierarchical storage module and the blockchain evidence storage module, and is used to perform block-based parallel compression and decompression processing on compliant data. The structure of the improved Merkle tree corresponds to the hot and cold data hierarchy of the hierarchical storage module. When data is migrated across levels, the root hash of the improved Merkle tree remains unchanged.
2. The blockchain-based trusted distributed data storage acceleration system according to claim 1, characterized in that, The process of constructing a trusted ledger for compliant data based on an improved Merkle tree structure, and generating a unique root hash covering all compliant data, specifically involves: Obtain all compliant data to be uploaded to the blockchain, and extract and mark four basic attributes for each piece of compliant data: compliance level, data subject, access frequency, and retention period. Based on the basic attributes of the compliant data that have been marked, an improved Merkle tree hierarchical structure corresponding to the cold and hot data hierarchy of the hierarchical storage module is constructed to complete the leaf node aggregation processing of the compliant data. Following the hierarchical structure of the improved Merkle tree, the aggregate hash calculation of each level of nodes is completed from bottom to top, generating the intermediate node hash of the corresponding level. Based on the aggregated hash of all hierarchical intermediate nodes, a unique global root hash covering all compliant data is generated, thus completing the construction of a trusted ledger for compliant data storage. When compliant data is migrated across levels, only the storage level to which the compliant data belongs is adjusted, without changing the hash value of each node in the improved Merkle tree, thus keeping the global root hash constant.
3. The blockchain-based trusted distributed data storage acceleration system according to claim 2, characterized in that, The hierarchical structure of the improved Merkle tree is constructed as follows: A leaf node layer is constructed to aggregate compliant data with the same compliance level and the same data subject into the same leaf node, and a single compliant data is not treated as an independent leaf node. An intermediate aggregation layer is constructed, which is divided into three levels of intermediate nodes according to the access frequency and retention period of compliant data. The three levels of intermediate nodes correspond one-to-one with the hot data, warm data and cold data levels of the tiered storage module. Each intermediate node stores the aggregate hash of all leaf nodes in the corresponding tier. A root node layer is constructed, and a unique global root hash is generated based on the aggregated hash of the three-level intermediate nodes. The global root hash covers the integrity information of all compliant data. Establish hierarchical binding rules to permanently bind the three-level intermediate nodes to the corresponding hot and cold data levels. The migration of compliant data across levels does not change the hash value of the corresponding node or its level.
4. The blockchain-based trusted distributed data storage acceleration system according to claim 3, characterized in that, The hash calculation and data integrity verification of the improved Merkle tree are specifically as follows: For fixed fields in compliant data, pre-hash caching is performed, and fixed fields of the same type of compliant data are hashed only once. Parallel hash calculations are performed on multiple aggregate nodes at the same level of the improved Merkle tree; For integrity verification of a single piece of compliant data, obtain the hash value of the leaf node to which the compliant data belongs, and complete the hash comparison verification along the path of the corresponding intermediate node; For integrity verification of batch compliant data in the same dimension, obtain the hash value of the corresponding aggregate leaf node of the batch of compliant data, and complete the hash comparison verification along a single path; For compliant data after cross-level migration, integrity verification is performed using the node hash path corresponding to the data before migration.
5. A blockchain-based trusted distributed data storage acceleration system according to claim 4, characterized in that, The specific steps for separating and storing hot and cold data according to the importance level and access frequency of compliant data are as follows: Establish data classification rules based on both compliance importance level and access frequency, dividing compliance data into three levels: hot data, warm data, and cold data. These three data levels correspond one-to-one with the three intermediate nodes of the improved Merkle tree. Establish the criteria for dividing each data level, and clarify the compliance level requirements, access frequency thresholds, and statutory retention period requirements for each level; Match the corresponding storage medium to each data level to complete the initial storage deployment of compliant data at each level; Based on the dynamic changes in the attributes of compliance data, perform cross-level migration of compliance data and synchronously update the storage location of compliance data; Configure the corresponding read / write access interface according to the storage tier to which the compliant data belongs.
6. The blockchain-based trusted distributed data storage acceleration system according to claim 5, characterized in that, The cross-level migration of execution data specifically refers to: Based on three preset trigger logics, it is determined whether the compliant data meets the conditions for cross-level migration. The three trigger logics include access frequency trigger, compliance cycle trigger, and capacity threshold trigger. For compliant data that meets the conditions for downgrade migration, perform migration operations from the hot data layer to the warm data layer, or from the warm data layer to the cold data layer. For compliant data that meets the upgrade and migration conditions, perform migration operations from the cold data layer to the warm data layer, or from the warm data layer to the hot data layer. For compliant data during the migration process, storage deployment is completed by matching the storage media corresponding to the target tier; After completing the cross-level migration of compliant data, the integrity of the migrated compliant data is verified by using the hash value of the corresponding node in the improved Merkle tree.
7. A blockchain-based trusted distributed data storage acceleration system according to claim 6, characterized in that, The specific steps for performing dynamic scheduling and on-demand data synchronization of lightweight nodes based on node load status are as follows: We construct a load assessment index system for lightweight nodes, setting four core assessment dimensions: the cache carrying capacity of the data at the corresponding node level, hash verification computing power, online stability, and task response latency. The system collects load data from all lightweight nodes in the scheduling pool in real time and calculates the overall health and task handling capacity of each node based on the load evaluation index system. Receive access and verification requests for compliant data, match available lightweight nodes at the corresponding level according to the storage level and operation type of the request, and assign corresponding processing tasks. Real-time monitoring of the load status and task execution of lightweight nodes; dynamic migration and load balancing adjustment of execution tasks when the node load exceeds a preset threshold. Based on the overall health changes of lightweight nodes, perform full lifecycle management of execution nodes.
8. A blockchain-based trusted distributed data storage acceleration system according to claim 7, characterized in that, The specific execution rules for the dynamic scheduling and on-demand data synchronization of the lightweight nodes are as follows: For lightweight nodes applying to join the scheduling pool, an admission verification is performed, which includes verifying the legitimacy of the node's identity and the resource carrying capacity of the corresponding level. For lightweight nodes that pass the admission verification, perform initial data synchronization. The synchronization content includes the global root hash of the improved Merkle tree and the intermediate node hash of the node's corresponding access level. After the synchronization is completed, the node is added to the scheduling pool to undertake processing tasks. For lightweight nodes that have been added to the scheduling pool, incremental data synchronization is performed. Incremental hash data and corresponding hot data are synchronized only when the hash of the intermediate node of the improved Merkle tree at the corresponding level is updated. For lightweight nodes that meet the exit conditions, a smooth exit process is executed. First, the cached data and processing tasks of the corresponding level of the node are migrated to other available nodes. After the handover is completed, the node is removed from the scheduling pool. For lightweight nodes that exit abnormally, disaster recovery measures are implemented, migrating the unfinished processing tasks and corresponding cached data of the node to a standby available node.
9. A blockchain-based trusted distributed data storage acceleration system according to claim 8, characterized in that, The specific steps for performing block-based parallel compression and decompression of compliant data are as follows: Based on the aggregation boundaries of the leaf nodes of the improved Merkle tree, independent compression blocks are divided, and aggregated compliant data within the same leaf node are divided into the same independent compression block; Each independent compressed block is assigned a separate computing thread to perform parallel compression processing, generating the corresponding compressed package and compressed block hash value; The hash value of the compressed block is included in the hash calculation range of the corresponding leaf node, thus completing the binding of the compressed data with the trusted evidence storage ledger; When receiving an access request for compliant data, decompression is performed only on the compressed block containing the target compliant data; During the decompression process, the integrity of compliant data is simultaneously verified using the hash value of the compressed block.
10. A blockchain-based trusted distributed data storage acceleration system according to claim 9, characterized in that, The algorithm adaptation and cooperative execution rules for the block-based parallel compression and decompression processing are as follows: For compliant data in the hot data layer, the LZ4 algorithm, modified for parallel processing, is used to perform compression and decompression. For compliant data in both the warm and cold data layers, a parallelized DEFLATE algorithm is used, along with a dedicated static dictionary for compliant data, to perform compression and decompression. The dedicated static dictionary for compliant data is pre-built based on fixed fields of the compliant data. During the compression process, the data stream of the compressed block is synchronously input into the compression thread and the hash calculation thread, and the compression process and hash calculation are executed synchronously. During the decompression process, the decompression data stream is synchronously input into the decompression thread and the hash verification thread, and the decompression process and integrity verification are executed synchronously. After compression or decompression, the integrity of all compliant data is verified by using the hash value of the corresponding node in the improved Merkle tree.